Method and apparatus for encoding/decoding image

ABSTRACT

A method and an apparatus for encoding/decoding an image according to the present invention may determine a reference area that is a pre-reconstructed area spatially adjacent to a current block on the basis of reference information for specifying the position of a reference area used for predicting a current block and may predict the current block on the basis of the reference area.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a division of application Ser. No. 17/043,121 filedon Sep. 29, 2020, which is a U.S. National Stage Application ofInternational Application No. PCT/KR2019/003737, filed on Mar. 29, 2019,which claims the benefit under 35 USC 119(a) and 365(b) of Korean PatentApplication No. 10-2018-0036296, filed on Mar. 29, 2018, and KoreanPatent Application No. 10-2018-0036297, filed on Mar. 29, 2018 in theKorean Intellectual Property Office, the entire disclosure of which isincorporated herein by reference for all purposes.

TECHNICAL FIELD

The present disclosure relates to video encoding/decoding method andapparatus.

BACKGROUND ART

Recently, demand for multimedia data such as moving pictures isincreasing rapidly in the Internet. However, development of a bandwidthof a channel may not keep up with the rapidly increasing amount of themultimedia data. Accordingly, VCEG (Video Coding Expert Group) of ITU-Tand Moving Picture Expert Group (MPEG) of ISO/IEC as an internationalstandardization organization have established a video compressionstandard HEVC (High Efficiency Video Coding) version 1 on February 2014.

The HEVC defines methods such as intra prediction, inter prediction,transform, quantization, entropy encoding, and an in-loop filter, etc.

DISCLOSURE Technical Problem

The present disclosure relates to video encoding/decoding method andapparatus. A purpose thereof is to provide inter/intra prediction methodand apparatus.

The present disclosure relates to video encoding/decoding method andapparatus. A purpose thereof is to provide block partitioning method andapparatus.

A purpose of the present disclosure is to provide a method and aapparatus of deriving motion information in a decoder.

Technical Solutions

A video encoding/decoding method and apparatus according to the presentdisclosure may determine reference information specifying a position ofa reference region used to predict a current block, determine thereference region which is a pre-reconstructed region spatially adjacentto the current block, based on the reference information, and predictthe current block based on the reference region.

In the video encoding/decoding method and apparatus according to thepresent disclosure, the reference region may be determined as one of aplurality of candidate regions, wherein the plurality of candidateregions may include at least one of a first candidate region, a secondcandidate region, or a third candidate region.

In the video encoding/decoding method and apparatus according to thepresent disclosure, the first candidate region may include a topneighboring region to the current block, the second candidate region mayinclude a left neighboring region to the current block, and the thirdcandidate region may include top and left neighboring regions to thecurrent block.

In the video encoding/decoding method and apparatus according to thepresent disclosure, the first candidate region may further include apartial region of a top-right neighboring region to the current block.

In the video encoding/decoding method and apparatus according to thepresent disclosure, the second candidate region may further include apartial region of a bottom-left neighboring region to the current block.

The video encoding/decoding method and apparatus according to thepresent disclosure may partition the current block into a plurality ofsub-blocks based on coding information of the current block, and maysequentially reconstruct the plurality of sub-blocks based onpre-determined priorities.

In the video encoding/decoding method and apparatus according to thepresent disclosure, the coding information may include first informationindicating whether the current block is partitioned.

In the video encoding/decoding method and apparatus according to thepresent disclosure, the coding information may further include secondinformation indicating a partitioning direction of the current block.

In the video encoding/decoding method and apparatus according to thepresent disclosure, the current block may be partitioned into two in ahorizontal direction or a vertical direction, based on the secondinformation.

The video encoding/decoding method and apparatus according to thepresent disclosure may derive a partitioning boundary point for theblock by using a block partitioning method using a motion boundary pointin the reconstructed region around the current block.

The video encoding/decoding method and apparatus according to thepresent disclosure may determine initial motion information of thecurrent block, determine delta motion information of the current block,and improve the initial motion information of the current block usingthe delta motion information, and perform motion compensation for thecurrent block using the improved motion information.

In the video encoding/decoding method and apparatus according to thepresent disclosure, determining the delta motion information may includedetermining a search region for improvement of the motion information,generating a SAD (sum of absolute difference) list from the searchregion, and updating the delta motion information based on a SADcandidate of the SAD list.

In the video encoding/decoding method and apparatus according to thepresent disclosure, the SAD list may specify a SAD candidate at eachsearch position in the search region.

In the video encoding/decoding method and apparatus according to thepresent disclosure, the search region may include a region extended by Nsample lines from a boundary of a reference block, wherein the referenceblock may be a region indicated by initial motion information of thecurrent block.

In the video encoding/decoding method and apparatus according to thepresent disclosure, the SAD candidate may be determined as a SAD valuebetween a L0 block and a L1 block, and the SAD value may be calculatedbased on some samples in the L0 and L1 blocks.

In the video encoding/decoding method and apparatus according to thepresent disclosure, a position of the L0 block may be determined basedon a position of a L0 reference block for the current block and apre-determined offset, wherein the offset may include at least one of anon-directional offset or a directional offset.

In the video encoding/decoding method and apparatus according to thepresent disclosure, the updating of the delta motion information isperformed based on a comparison result between a reference SAD candidateand a pre-determined threshold, wherein the reference SAD candidate maymean a SAD candidate corresponding to the non-directional offset.

In the video encoding/decoding method and apparatus according to thepresent disclosure, when the reference SAD candidate is greater than orequal to the threshold, a SAD candidate with a minimum value among theSAD candidates in the SAD list is identified, and the delta motioninformation may be updated based on an offset corresponding to theidentified SAD candidate.

In the video encoding/decoding method and apparatus according to thepresent disclosure, when the reference SAD candidate is smaller than thethreshold, the delta motion information may be updated based on aparameter calculated using all or some of the SAD candidates included inthe SAD list.

In the video encoding/decoding method and apparatus according to thepresent disclosure, the improvement of the initial motion informationmay be performed on a sub-block basis in consideration of a size of thecurrent block.

In the video encoding/decoding method and apparatus according to thepresent disclosure, the improvement of the initial motion informationmay be selectively performed based on at least one of a size of thecurrent block, a distance between a current picture and a referencepicture, an inter prediction mode, a prediction direction, or aresolution of the motion information.

TECHNICAL EFFECTS

According to the present disclosure, the method and apparatus mayimprove accuracy of the prediction via template block-based prediction,and improve encoding efficiency.

According to the present disclosure, encoding efficiency may be improvedby performing prediction on a sub-block basis via adaptive blockpartitioning.

According to the present disclosure, the method and apparatus may moreaccurately detect a partitioning boundary point of a block using amotion boundary point between different objects within the reconstructedregion, such that compression efficiency may be improved.

According to the present disclosure, an accuracy of inter prediction maybe improved via the improvement of the motion information.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a simplified flowchart of a video encoding apparatus.

FIG. 2 is a diagram for illustrating in detail a block partitioning unitof a video encoding apparatus.

FIG. 3 is a diagram for illustrating in detail a prediction unit of avideo encoding apparatus.

FIG. 4 is a diagram for illustrating a motion estimator of a predictionunit in a video encoding apparatus.

FIG. 5 is a flowchart showing a method of deriving candidate motioninformation in skip and merge modes.

FIG. 6 is a flowchart showing a method of deriving candidate motioninformation in an

AMVP mode.

FIG. 7 is a diagram showing positions of reconstructed blocks used toderive spatial/temporal candidates among candidate motion information inskip, merge, and AMVP modes.

FIG. 8 is a diagram to illustrate a method of deriving a temporalcandidate among candidate motion information in skip, merge, and AMVPmodes.

FIG. 9 is a diagram to illustrate a method of deriving a combinedbi-directional candidate mode among candidate motion information in skipand merge modes.

FIG. 10 is a diagram showing motion estimated patterns used in a motionestimator in a video encoding apparatus or a DMVD motion estimator in avideo encoding/decoding apparatus according to one embodiment.

FIG. 11 is a flowchart showing a method of encoding partitioninginformation and prediction information.

FIG. 12 is a schematic flowchart showing a video decoding apparatus.

FIG. 13 is a diagram for illustrating a block partitioning unit of avideo decoding apparatus.

FIG. 14 is a diagram for illustrating a prediction unit of a videodecoding apparatus.

FIG. 15 is a flowchart showing a method of decoding partitioninginformation and prediction information.

FIG. 16 is a diagram for illustrating a block partitioning unit of avideo encoding apparatus according to one embodiment of the presentdisclosure.

FIG. 17 is a diagram for illustrating a block partitioning unit of avideo decoding apparatus according to one embodiment of the presentdisclosure.

FIG. 18 is a diagram for illustrating a prediction unit of a videoencoding apparatus according to one embodiment of the presentdisclosure.

FIG. 19 is a diagram for illustrating a prediction unit of a videodecoding apparatus according to one embodiment of the presentdisclosure.

FIG. 20 is a diagram for illustrating a DMVD motion estimator of aprediction unit in a video encoding/decoding apparatus according to oneembodiment of the present disclosure.

FIG. 21 is a diagram for illustrating a DMVD mode according to oneembodiment of the present disclosure.

FIG. 22 is a diagram for illustrating a method of determining a templateblock in a reconstructed region in a DMVD mode according to oneembodiment of the present disclosure.

FIG. 23 is a diagram for illustrating a method of searching for apartitioning motion boundary point for a block according to oneembodiment of the present disclosure.

FIG. 24 is a diagram for illustrating a fast algorithm for searching fora motion boundary point in FIG. 22 .

FIG. 25 is a diagram for illustrating a DMVD initial motion informationdetector in a prediction unit in a video encoding/decoding apparatusaccording to one embodiment of the present disclosure.

FIG. 26 is a diagram to illustrate a method of calculating a cost valueper line in the fast algorithm of FIG. 23 .

FIG. 27 is a diagram for illustrating an example of a block that isdivided into two or four based on one or two partitioning motionboundary points for the block according to one embodiment of the presentdisclosure.

FIG. 28 is a diagram showing a method of searching for motioninformation using only a partial region adjacent to a current block inthe reconstructed region as a template based on a partitioning motionboundary point of FIG. 23 and FIG. 24 .

FIG. 29 is a diagram showing a method of searching motion information byusing the entire region adjacent to the current block in thereconstructed region as a template based on the partitioning motionboundary points of FIGS. 23 and 24 .

FIG. 30 is a flowchart illustrating a method of encoding partitioninginformation and prediction information according to one embodiment ofthe present disclosure.

FIG. 31 is a flowchart illustrating a method of decoding partitioninginformation and prediction information according to one embodiment ofthe present disclosure.

FIG. 32 shows an example of partitioning a coding block according to oneembodiment of the present disclosure.

FIG. 33 is a diagram for illustrating a DMVD motion estimator of aprediction unit in a video encoding/decoding apparatus according to oneembodiment of the present disclosure.

FIG. 34 is a flow chart showing a method of determining a slope of amotion boundary line according to one embodiment of the presentdisclosure.

FIG. 35 is a diagram to illustrate a method of determining a slope of amotion boundary line in a motion boundary search region of FIG. 34 andpartitioning a current block.

FIG. 36 is a diagram for illustrating a filtering method in partitioninga current block along a straight line according to one embodiment of thepresent disclosure.

FIG. 37 is a diagram for illustrating a filtering method in partitioninga current block along an oblique line according to one embodiment of thepresent disclosure.

FIG. 38 is a flowchart to illustrate a method of encoding predictioninformation according to one embodiment of the present disclosure.

FIG. 39 is a flow chart illustrating a method of decoding predictioninformation according to one embodiment of the present disclosure.

FIG. 40 shows a method for performing intra prediction in a predictionunit of a video encoding/decoding apparatus in an embodiment to whichthe present disclosure is applied.

FIG. 41 shows an intra prediction method on a sub-block basis in anembodiment to which the present disclosure is applied.

FIG. 42 shows a prediction method based on inter-components reference inan embodiment to which the present disclosure is applied.

FIG. 43 shows a method of determining delta motion information in anembodiment to which the present disclosure is applied.

BEST MODE OF DISCLOSURE

A video encoding/decoding method and apparatus according to the presentdisclosure may determine reference information specifying a position ofa reference region used to predict a current block, determine thereference region which is a pre-reconstructed region spatially adjacentto the current block, based on the reference information, and predictthe current block based on the reference region.

In the video encoding/decoding method and apparatus according to thepresent disclosure, the reference region may be determined as one of aplurality of candidate regions, wherein the plurality of candidateregions may include at least one of a first candidate region, a secondcandidate region, or a third candidate region.

In the video encoding/decoding method and apparatus according to thepresent disclosure, the first candidate region may include a topneighboring region to the current block, the second candidate region mayinclude a left neighboring region to the current block, and the thirdcandidate region may include top and left neighboring regions to thecurrent block.

In the video encoding/decoding method and apparatus according to thepresent disclosure, the first candidate region may further include apartial region of a top-right neighboring region to the current block.

In the video encoding/decoding method and apparatus according to thepresent disclosure, the second candidate region may further include apartial region of a bottom-left neighboring region to the current block.

The video encoding/decoding method and apparatus according to thepresent disclosure may partition the current block into a plurality ofsub-blocks based on coding information of the current block, and maysequentially reconstruct the plurality of sub-blocks based onpre-determined priorities.

In the video encoding/decoding method and apparatus according to thepresent disclosure, the coding information may include first informationindicating whether the current block is partitioned.

In the video encoding/decoding method and apparatus according to thepresent disclosure, the coding information may further include secondinformation indicating a partitioning direction of the current block.

In the video encoding/decoding method and apparatus according to thepresent disclosure, the current block may be partitioned into two in ahorizontal direction or a vertical direction, based on the secondinformation.

The video encoding/decoding method and apparatus according to thepresent disclosure may derive a partitioning boundary point for theblock by using a block partitioning method using a motion boundary pointin the reconstructed region around the current block.

The video encoding/decoding method and apparatus according to thepresent disclosure may determine initial motion information of thecurrent block, determine delta motion information of the current block,and improve the initial motion information of the current block usingthe delta motion information, and perform motion compensation for thecurrent block using the improved motion information.

In the video encoding/decoding method and apparatus according to thepresent disclosure, determining the delta motion information may includedetermining a search region for improvement of the motion information,generating a SAD (sum of absolute difference) list from the searchregion, and updating the delta motion information based on a SADcandidate of the SAD list.

In the video encoding/decoding method and apparatus according to thepresent disclosure, the SAD list may specify a SAD candidate at eachsearch position in the search region.

In the video encoding/decoding method and apparatus according to thepresent disclosure, the search region may include a region extended by Nsample lines from a boundary of a reference block, wherein the referenceblock may be a region indicated by initial motion information of thecurrent block.

In the video encoding/decoding method and apparatus according to thepresent disclosure, the SAD candidate may be determined as a SAD valuebetween a L0 block and a L1 block, and the SAD value may be calculatedbased on some samples in the L0 and L1 blocks.

In the video encoding/decoding method and apparatus according to thepresent disclosure, a position of the L0 block may be determined basedon a position of a L0 reference block for the current block and apre-determined offset, wherein the offset may include at least one of anon-directional offset or a directional offset.

In the video encoding/decoding method and apparatus according to thepresent disclosure, the updating of the delta motion information isperformed based on a comparison result between a reference SAD candidateand a pre-determined threshold, wherein the reference SAD candidate maymean a SAD candidate corresponding to the non-directional offset.

In the video encoding/decoding method and apparatus according to thepresent disclosure, when the reference SAD candidate is greater than orequal to the threshold, a SAD candidate with a minimum value among theSAD candidates in the SAD list is identified, and the delta motioninformation may be updated based on an offset corresponding to theidentified SAD candidate.

In the video encoding/decoding method and apparatus according to thepresent disclosure, when the reference SAD candidate is smaller than thethreshold, the delta motion information may be updated based on aparameter calculated using all or some of the SAD candidates included inthe SAD list.

In the video encoding/decoding method and apparatus according to thepresent disclosure, the improvement of the initial motion informationmay be performed on a sub-block basis in consideration of a size of thecurrent block.

In the video encoding/decoding method and apparatus according to thepresent disclosure, the improvement of the initial motion informationmay be selectively performed based on at least one of a size of thecurrent block, a distance between a current picture and a referencepicture, an inter prediction mode, a prediction direction, or aresolution of the motion information.

Embodiments Of Disclosure

Embodiments of the present disclosure will be described in detail withreference to the drawings attached to the present disclosure so thatthose of ordinary skill in the technical field to which the presentdisclosure belongs may easily implement the disclosure. However, thepresent disclosure may be implemented in various different forms, andmay not be limited to the exemplary embodiments illustrated herein. Inaddition, in the drawings, in order to clearly illustrate the presentdisclosure, parts irrelevant to the description are omitted. In thedrawings, similar reference numerals are allocated to similar partsthroughout the specification.

It will be understood that when an element is referred to as being“connected to” another element, it may be directly on, connected to theother element, or one or more intervening elements may be present.

In the entire specification, the meaning that a certain element includesthe other element does not exclude other components, but means thatother components may be further included unless specifically stated tothe contrary.

It will be understood that, although the terms “first”, “second”, and soon may be used herein to describe various elements, these elementsshould not be limited by these terms. These terms are used todistinguish one element from another element.

Further, in the embodiments of the apparatus and the method as describedin the present disclosure, some of components of the apparatus or someof steps of the method may be omitted. Some of components of theapparatus or some of steps of the method may vary in terms of anarrangement order. In addition, some of components of the apparatus orsome of steps of the method may be replaced with other components orother steps.

Further, some components or some steps of a first embodiment of thepresent disclosure may be added to a second embodiment of the presentdisclosure, or may replace some components or some steps of the secondembodiment.

In addition, components shown in the embodiment of the presentdisclosure are shown independently to represent different characteristicfunctions. This does not mean that each of the components constitutesindividual hardware or software. That is, each of the component isindividually described for convenience of description. At least twocomponents may be combined into a single component, or a singlecomponent may be divided into a plurality of components. An embodimentin which at least two components are combined into a single componentand an embodiment in which a single component is divided into aplurality of components may be included in a scope of the presentdisclosure as long as they do not deviate from the essence of thepresent disclosure.

Hereinafter, exemplary embodiments of the present disclosure will beillustrated in more detail with reference to the accompanying drawings.In describing the present disclosure, duplicate descriptions about thesame components are omitted.

FIG. 1 is a block flow diagram schematically showing a configuration ofa video encoding apparatus. The video encoding apparatus is configuredto encode video data. The video encoding apparatus may include a blockpartitioning unit, a prediction unit, a transform unit, a quantizationunit, an entropy encoding unit, a dequantization unit, aninverse-transform unit, an adder, an in-loop filter, a memory, and asubtractor.

The block partitioning unit 101 partitions from a coding block of themaximum size (hereinafter referred to maximum coding block) to a codingblock of the minimum size (hereinafter referred to minimum codingblock). There are various block partitioning methods. A quad-treepartitioning (hereinafter referred to as a QT partitioning) partitions acurrent coding block into four blocks with the same size. A binary-treepartitioning (hereinafter referred to as a BT partitioning) partitions acoding block into two blocks with the same size in a horizontaldirection or a vertical direction. There are various partitioningmethods other than the QT and BT partitioning methods. Further, variouspartitioning methods may be concurrently applied.

FIG. 2 shows an operation flow of a block partitioning unit in a videoencoding apparatus. As a partitioning method of a higher depth block,one of QT partitioning 201 and BT partitioning 203 may be selected. Whenthe QT partitioning is performed on the higher depth block, a currentblock may be determined by generating a lower depth block throughdividing the higher depth block into four 202. When the BT partitioningis performed on the higher depth block, a current block may bedetermined by generating a lower depth block through dividing the higherdepth block into two 204.

The prediction unit 102 generates a prediction block of a currentoriginal block using pixels adjacent to a block (hereinafter referred toas a prediction block) to be currently predicted or pixels in areference picture that has already been encoded/decoded. One or moreprediction blocks may be generated within the coding blockcoding block.When there is one prediction block in the coding block, the predictionblock has the same shape as the coding block. A video signal predictionis largely composed of intra prediction and inter prediction. In theintra prediction, a prediction block is generated using neighboringpixels to the current block. In the inter prediction, a prediction blockis generated using a block that is most similar to the current block ina reference picture that has already been encoded/decoded. Then, aresidual block may be determined as a difference between the originalblock and the prediction block. The residual block may be subjected tovarious techniques such as rate-distortion optimization (RDO) todetermine an optimal prediction mode of the prediction block. The RDOcost calculation formula is the same as in Equation 1.

J(Φ, λ)=D(Φ)+λR(Φ)   [Equation 1]

D, R, and J indicate a deterioration due to quantization, a rate of acompressed stream, and a RD cost, respectively. Φ is an encoding mode. Ais a Lagrangian multiplier, and is used as a scale correctioncoefficient to match an unit of an error amount with an unit of a bitamount. In order to select an optimal encoding mode in a encodingprocess, J when the optimal encoding mode is applied, that is, a RD-costvalue must be smaller than that when other encoding modes are applied.The RD-cost value is calculated while simultaneously considering a bitrate and an error.

FIG. 3 is a flowchart illustrating an operation flow in a predictionunit of a video encoding apparatus. When performing intra predictionusing original information and reconstructed information (301), anoptimal intra prediction mode for each prediction mode is determinedusing the RD-cost value (302), and the prediction is generated. Whenperforming inter prediction using reconstructed information (303), theRD-cost value is calculated for a skip mode, a merge mode, and an AMVPmode. A merge candidate detector 304 configures a set of candidatemotion information for the skip mode and the merge mode. Among thecandidate motion information set, optimal motion information isdetermined using the RD-cost value (305). An AMVP candidate detector 306configures a set of candidate motion information for the AMVP mode.Motion estimation is performed using the set of the candidate motioninformation to determine optimal motion information (307). A predictionblock is generated by performing motion compensation using optimal themotion information determined in each mode (308).

FIG. 4 illustrates an operation flow of a motion estimator of theprediction unit in the video encoding apparatus. A motion estimationstart point in a reconstructed picture is determined using candidatemotion information of the AMVP mode (401). In this case, the motionestimation start point may not be determined using AMVP candidate motioninformation (that is, this information means Motion Vector Predictor).However, the motion estimation may be started at a pre-set arbitrarystart point. A search maximum/minimum precision (N, M) of a motionvector for motion estimation is determined (402), and a motion estimatedpattern at a current estimation precision N is determined, and motionestimation is performed (403). Here, FIG. 10 is referred to for themotion estimated pattern. FIG. 10 shows examples of four motionestimated patterns. A pixel point marked with R in each example means apixel point currently being estimated. A pixel point marked with S1refers to a pixel point that is under the first step motion-estimationfor each pattern among pixel points adjacent to the R pixel. A pixelpoint marked with S2 refers to adjacent pixel points of the same patternto a pixel point having the smallest motion estimation cost value amongS1 pixel points. In this way, the motion estimation may be repeatedlyperformed up to a point SL (L=1, 2, . . . ). After performing the motionestimation, the current motion estimation precision N is refined (404).When the refined precision N becomes smaller than a minimum estimationprecision M, the optimal motion information corresponding to up to thecurrent precision is determined as the optimal motion information of thecurrent block, and this flowchart is terminated. Otherwise, theflowchart returns to step 403 and the above process is repeated.

The above-described inter prediction may be composed of three modes(skip mode, merge mode, AMVP mode). Each prediction mode obtains theprediction block of the current block using motion information(prediction direction information, reference picture information, motionvector), and an additional prediction mode using the motion informationmay be present.

The skip mode determines optimal prediction information using motioninformation of an pre-reconstructed region. A motion informationcandidate group is configured within the reconstructed region, and aprediction block is generated using a candidate with the lowest RD-costvalue among the candidate group as prediction information. Here, amethod of configuring the motion information candidate group is the sameas a method of configuring the motion information candidate group in themerge mode to be described below and thus descriptions thereof will beomitted.

The merge mode is the same as the skip mode in that the optimalprediction information is determined using motion information of thepre-reconstructed region. However, in the skip mode, motion informationthat allows an prediction error to be 0 is searched in the motioninformation candidate group, whereas in the merge mode, motioninformation that allows the prediction error not to be 0 is searched inthe motion information candidate group. In a manner similar to the skipmode, in the merge mode, a motion information candidate group isconfigured within the reconstructed region, and a prediction block isgenerated using a candidate with the lowest RD-cost value among thecandidate group as prediction information.

FIG. 5 shows a method of generating a motion information candidate groupin the skip mode and the merge mode. The maximum number of motioninformation candidates may be determined equally in the video encodingapparatus and the video decoding apparatus. The number information maybe previously transmitted in a higher header of the video encodingapparatus (the higher header means parameters transmitted from a higherlayer of a block such as a video parameter layer, a sequence parameterlayer, and a picture parameter layer). In the description of step S501and step S502, only when a spatial candidate block and a temporalcandidate block are encoded in an inter prediction mode, the motioninformation derived using the corresponding motion information isincluded in the motion information candidate group. In step S501, 4candidates of 5 spatial candidate blocks around the current block areselected within the same picture. FIG. 7 is referred to for the positionof the spatial candidate. The position of each candidate may be changedto any block within the reconstructed region. The spatial candidate isconsidered in an order of A₁, A₂, A₃, A₄, and A₅, and motion informationof the first available spatial candidate block is determined as aspatial candidate. When there are duplicate motion information, onlymotion information of a candidate with high priority is considered. Instep S502, one candidate of two temporal candidate blocks is selected.FIG. 7 is referred to for the position of the temporal candidate. Aposition of each candidate is determined based on a block of the sameposition in a collocated picture as a position of the current block inthe current picture. Here, the video encoding apparatus and the videodecoding apparatus may set the collocated picture among reconstructedpictures with the same conditions. It is considered in an order of B₁and B₂ blocks, and motion information of the first available candidateblock is determined as a temporal candidate. FIG. 8 is referred to for amethod to determine the motion information of the temporal candidate.The motion information of each of the candidate blocks B₁ and B₂ in thecollocated picture indicates a prediction block in a reference pictureB. (However, the reference picture of each candidate block may bedifferent from each other. In this description, all these referencepictures are referred to as the reference picture B for convenience.) Aratio of a distance between the current picture and the referencepicture A to a distance between the collocated picture and the referencepicture B is calculated. The motion vector of the candidate block isscaled by the calculated ratio to determine the motion vector of thetemporal candidate motion information. Equation 2 refers to the scalingequation.

$\begin{matrix}{{ {{ {{{{MV}_{scale} = ( ( ( {{{TB} \times \frac{ {{{2^{14} + ( {TD} }}1} )}{TD}} + 2^{5}} )  }}6} ) \times {MV}} + ( {2^{7} - 1} )} )}2^{3}} & \lbrack {{Equation}2} \rbrack\end{matrix}$

MV denotes a motion vector of temporal candidate block motioninformation, MV_(scale) denotes a scaled motion vector, TB denotes atemporal distance between the collocated picture and the referencepicture B, and TD denotes a temporal distance between the currentpicture and the reference picture A. Further, the reference picture Aand the reference picture B may be the same reference picture. Thisscaled motion vector is determined as a motion vector of the temporalcandidate, and reference picture information of the temporal candidatemotion information is determined as a reference picture of the currentpicture, thereby to derive the motion information of the temporalcandidate. Step S503 is performed only when the maximum number of motioninformation candidates is not satisfied in steps S501 and S502. Thisstep S503 is to add a new bi-directional motion information candidate asa combination of motion information candidates derived from the previousstep. The bi-directional motion information candidate refers to a newcandidate combination of motion information in past or future directionsderived previously. A table in FIG. 9 shows the priorities ofbi-directional motion information candidate combinations. In addition tothe combinations in this table, additional combinations may be present.This table shows an example. When the maximum number of motioninformation candidates is not satisfied even when the bi-directionalmotion information candidate is used, step S504 is performed. In stepS504, the maximum number of motion information candidates is satisfiedby fixing the motion vector of the motion information candidate as azero motion vector and varying the reference picture based on theprediction direction.

In the AMVP mode, optimal motion information is determined via motionestimation for each reference picture based on the prediction direction.Here, the prediction direction may be an uni-direction including onlyone direction of past/future directions or may be bi-direction includingboth the past and future directions. A prediction block is generated byperforming motion compensation using optimal motion informationdetermined via motion estimation. Here, a motion information candidategroup for motion estimation is derived for each reference pictureaccording to the prediction direction. The motion information candidategroup is used as a starting point for motion estimation. FIG. 6 may bereferred to for a method of deriving a motion information candidategroup for motion estimation in the AMVP mode.

The maximum number of motion information candidates may be determinedequally in the video encoding apparatus and the video decodingapparatus. The number information may be previously transmitted in thehigher header of the video encoding apparatus. Only when the spatialcandidate block and temporal candidate block are encoded in an interprediction mode in the description of step S601 and step S602, themotion information derived using the corresponding motion information isincluded in the motion information candidate group. In step S601, unlikethe description of step S501 in FIG. 5 , the number of derived spatialcandidates (two) may vary, and priorities for selecting spatialcandidates may also vary. The rest of the description is the same asthat of step S501. Description of step S602 is the same as thedescription of step S502. In step S603, when there is duplicate motioninformation among candidates derived so far, the duplicate motioninformation is removed. Step S604 is the same as the description of stepS504. The motion information candidate with the minimum RD-cost valueamong the motion information candidates derived in this way is selectedas the optimal motion information candidate, and the optimal motioninformation of the AMVP mode is obtained via a motion estimation processbased on the corresponding motion information.

The transform unit 103 generates a transform block by transforming aresidual block of a difference between the original block and theprediction block. The transform block is the smallest unit used for thetransform and quantization processes. The transform unit transforms theresidual signal into a frequency region to generate a transform blockhaving transform coefficients. Various transform methods such as adiscrete cosine transform (DCT), a discrete sine transform (DS), and KLT(Karhunen Loeve Transform) may be used as a method for transforming theresidual signal into the frequency region may include. The transformcoefficient is generated by transforming, based on these method, theresidual signal into the frequency region. In order to conveniently usethe transform method, a matrix operation is performed using a basisvector. Depending on which prediction mode in which the prediction blockis encoded, transform methods may be used together during the matrixoperation. For example, when performing the intra prediction, a discretecosine transform may be used in the horizontal direction and a discretesine transform may be used in the vertical direction, based on theprediction mode.

The quantization unit 104 quantizes the transform block to generate aquantized transform block. That is, the quantization unit quantizestransform coefficients of the transform block generated from thetransform unit 103 to generate a quantized transform block havingquantized transform coefficients. Dead zone uniform thresholdquantization (DZUTQ) or a quantization weighted matrix may be used asthe quantization method. However, various quantization methods such asimproved quantization thereof may be used.

In the above description, it is illustrated and described that the videoencoding apparatus includes the transform unit and the quantizationunit. However, the transform unit and the quantization unit may beoptionally included therein. That is, the video encoding apparatus maytransform the residual block to generate the transform block and may notperform the quantization process. Alternatively, the video encodingapparatus may not transform the residual block into a frequencycoefficient but may perform only the quantization process.Alternatively, the video encoding apparatus may not perform both of thetransform and quantization processes. Although the video encodingapparatus does not perform the transforming process and/or thequantization process, a block input to an input of the entropy encodingunit is commonly referred to as a ‘quantized transform block’.

The entropy encoding unit 105 encodes the quantized transform block andoutputs a bitstream. That is, the entropy encoding unit encodes thecoefficients of the quantized transform block output from thequantization unit using various encoding methods such as the entropyencoding method. The entropy encoding unit 105 generates and outputs abitstream including additional information (for example, information onthe prediction mode (the information on the prediction mode may includemotion information or intra prediction mode information determined bythe prediction unit), quantization coefficient, etc.) necessary todecode the corresponding block in the video decoding apparatus to bedescribed later.

The dequantization unit 106 reconstructs a dequantized transform blockby performing the quantization method used for quantization on thequantized transform block in reverse.

The inverse-transform unit 107 reconstructs the residual block byinverse transforming the dequantized transform block using the samemethod as the method used for transform. That is, the transform methodused in the transform unit is performed in reverse.

In the above description, the dequantization unit and theinverse-transform unit may perform dequantization and inverse transformby using the quantization method and the transform method respectivelyused in the quantization unit and the transform unit in reverse.Alternatively, when only quantization is performed but no transform isperformed in the transform unit and the quantization unit, onlydequantization may be performed and inverse transform may not beperformed. Alternatively, when neither transform nor quantization isperformed in the quantization unit and the transform unit, thedequantization unit and the inverse-transform unit may not perform bothinverse transform and dequantization or may be omitted and not beincluded in the video encoding apparatus.

The adder 108 reconstructs the current block by adding the residualsignal generated by the inverse-transform unit and the prediction blockgenerated via prediction.

The filter 109 may additionally filter an entire picture after allblocks in the current picture are reconstructed. The filter may performfiltering such as deblocking filtering and SAO (Sample Adaptive Offset).Deblocking filtering refers to a process of reducing block distortionthat occurs while encoding the video data on a block basis. SAO (SampleAdaptive Offset) refers a process of minimizing a difference between thereconstructed video and the original video by subtracting or adding aspecific value from or to a reconstructed pixel.

The memory 110 stores the reconstructed current block obtained viaadding the residual signal generated by the inverse-transform unit tothe prediction block generated via the prediction, and then viaadditional filtering using the in-loop filter. The reconstructed currentblock may be used to predict a next block or next picture.

The subtractor 111 generates the residual block by subtracting theprediction block from the current original block.

FIG. 11 is a flowchart showing an encoding flow of coding information ina video encoding apparatus. In step S1101, information indicatingwhether the current coding block is partitioned is encoded. In stepS1102, it is determined whether the current coding block is partitioned.When it is determined that the current coding block is partitioned instep S1102, in step S1103, information on which partitioning method isused among QT partitioning, BT horizontal partitioning, and BT verticalpartitioning is encoded. In step S1104, the current coding block ispartitioned based on the partitioning method. In step S1105, aftermoving to the first sub coding block partitioned within the currentcoding block, the process returns to step S1101. When it is determinedthat the current coding block is not partitioned in step S1102,information about activation of the skip mode is encoded in step S1106.In step S1107, it is determined whether skip mode is activated. When theskip mode is activated in step S1107, a merge candidate indexinformation for the skip mode is encoded in step S1113. Thereafter, theprocess moves to step S1123. The description thereof will be describedin detail below. When the skip mode is not activated in step S1107, theprediction mode is encoded in step S1108. In step S1109, it isdetermined whether the prediction mode is inter prediction or intraprediction mode. When the prediction mode is intra prediction mode instep S1109, intra prediction mode information is encoded in step S1110and the process moves to step S1123. The description thereof will bedescribed in detail below. When the prediction mode is inter predictionmode in step S1109, the information about activation of the merge modeis encoded in step S1111. In step S1112, it is determined whether themerge mode is activated. When the merge mode is activated in step S1112,the process moves to step S1113 to encode merge candidate indexinformation for the merge mode. When the merge mode is not activated instep S1112, the prediction direction is encoded in step S1114. Here,prediction direction may be one of a past direction, a future direction,and a bi-direction. In step S1115, it is determined whether theprediction direction is the future direction. When the predictiondirection is not the future direction in step S1115, the past referencepicture index information is encoded in step S1116. In step S1117, MVD(Motion Vector Difference) information of the past direction is encoded.In step S1118, motion vector predictor (MVP) information of the pastdirection is encoded. When in step S1115, the prediction direction isthe future direction or bi-directional, or when step S1118 is finished,whether the prediction direction is the past direction is determined instep S1119. When the prediction direction is not the past direction instep S1119, the reference picture index information of the futuredirection is encoded in step S1120. In step S1121, MVD information ofthe future direction is encoded. In step S1122, MVP information of thefuture direction is encoded. When, in step S1119, the predictiondirection is the past direction or bi-directional, or when step S1122 isfinished, it is determined in step S1123 whether encoding of all subcoding blocks is ended. Here, the process of this step is performed evenafter step S1113 is finished. When finished, the flowchart isterminated. When not finished, the process moves from a current subcoding block to a next sub coding block in step S1124, and theabove-described process is repeated from step S1101.

FIG. 12 is a block flow diagram briefly showing a configuration of avideo decoding apparatus. The video decoding apparatus is an apparatusof decoding video data, and may largely include a block entropy decodingunit, a dequantization unit, an inverse-transform unit, a predictionunit, an adder, an in-loop filter, and a memory. A coding block in thevideo encoding apparatus is referred to as a decoding block in the videodecoding apparatus.

The entropy decoding unit 1201 interprets the bitstream transmitted fromthe video encoding apparatus to read out various information and thequantized transform coefficient that is necessary for decoding theblock.

The dequantization unit 1202 reconstructs a dequantized block having adequantized coefficient by performing the quantization method used forquantization in reverse on the quantization coefficient decoded by theentropy decoding unit.

The inverse-transform unit 1203 reconstructs the residual block having adifference signal by inverse-transforming the dequantized transformblock using the same method as the method used for transform. Thetransform method used in the transform unit is performed in reverse.

The prediction unit 1204 generates a prediction block using predictionmode information decoded by the entropy decoding unit. Here, theprediction unit 1204 uses the same method as the prediction methodperformed by the prediction unit of the video encoding apparatus.

The adder 1205 reconstructs the current block by adding the residualsignal reconstructed in the inverse-transform unit to the predictionblock generated via the prediction.

The filter 1206 performs additional filtering on an entire picture afterreconstructing all blocks in the current picture, and the filteringincludes deblocking filtering and SAO (Sample Adaptive Offset). Detailsthereof are the same as described with reference to the in-loop filterof the video encoding apparatus as described above.

The memory 1207 stores the reconstructed current block obtained viaadding the residual signal generated by the inverse-transform unit tothe prediction block generated via the prediction, and then viaadditional filtering using the in-loop filter. The reconstructed currentblock may be used to predict a next block or next picture.

FIG. 13 shows an operation flow of the block partitioning unit in avideo decoding apparatus. Partitioning information is extracted from ahigher depth block layer (1301), and a partitioning method is selectedfrom QT partitioning (1302) or BT partitioning (1304). When QTpartitioning is performed, the current block is determined by generatinglower depth blocks through quatering (1303) a higher depth block. WhenBT partitioning is performed, the current block is determined bygenerating lower depth blocks through bisecting (1305) a higher depthblock.

FIG. 14 is a flow chart illustrating an operation flow in a predictionunit of a video decoding apparatus. When the prediction mode is an intraprediction mode, a prediction block is generated by determining optimalintra prediction mode information (1401) and performing intra prediction(1402). When the prediction mode is an inter prediction mode, an optimalprediction mode is determined among skip, merge, and AMVP modes (1403).When decoding is performed in the skip mode or the merge mode, a mergecandidate detector configures a set of candidate motion information forthe skip mode and the merge mode (1404). Among the set of candidatemotion information, optimal motion information is determined (1405).When it is decoded in the AMVP mode, an AMVP candidate detectorconfigures a set of candidate motion information for the AMVP mode(1406). Among the candidate motion information candidates, optimalmotion information is determine based on the transmitted MVP information(1407). Thereafter, motion compensation is performed using the optimalmotion information determined in each mode to generate a predictionblock (1408).

FIG. 15 is a flow chart showing a decoding flow of coding information ina video decoding apparatus. In step S1501, information indicatingwhether the current decoding block is partitioned is decoded. In stepS1502, it is determined whether the current decoding block ispartitioned. When the current decoding block is partitioned in stepS1502, information about which partitioning method is used among QTpartitioning, BT horizontal partitioning, and BT vertical partitioningis decoded in step S1503. In step S1504, the current decoding block ispartitioned according to the partitioning method. In step S1505, aftermoving to the first sub decoding block partitioned within the currentdecoding block, the process returns to step S1501. When the currentdecoding block is not partitioned in step S1502, information aboutactivation of the skip mode is decoded in step S1506. In step S1507,whether the skip mode is activated is determined. When the skip mode isactivated in step S1507, the merge candidate index information for theskip mode is decoded in step S1513. Then, the process moves to stepS1523. The description thereof will be described in detail below. Whenthe skip mode is not activated in step S1507, the prediction mode isdecoded in step S1508. In step S1509, it is determined whether theprediction mode is an inter prediction or an intra prediction mode. Whenthe prediction mode is the intra prediction mode in step S1509, theintra prediction mode information is decoded in step S1510. The processmoves to step S1523. The description thereof will be described in detailbelow. When the prediction mode is the inter prediction mode in stepS1509, the information about activation of the merge mode is decoded instep S1511. In step S1512, it is determined whether the merge mode isactivated. When the merge mode is activated in step S1512, the processmoves to step S1513 where merge candidate index information for themerge mode is decoded. When the merge mode is not activated in stepS1512, the prediction direction is decoded in step S1514. Here, theprediction direction may be one of a past direction, a future direction,and a bi-direction. In step S1515, it is determined whether theprediction direction is the future direction. When the predictiondirection is not the future direction in step S1515, the referencepicture index information of the past direction is decoded in stepS1516. In step S1517, MVD (Motion Vector Difference) information of thepast direction is decoded. In step S1518, motion vector predictor (MVP)information of the past direction is decoded. When, in step S1515, theprediction direction is the future direction or the bi-direction, orwhen the step S1518 is finished, it is determined whether the predictiondirection is the past direction in step S1519. When the predictiondirection is not the past direction in step S1519, the reference pictureindex information of the future direction is decoded in step S1520. Instep S1521, MVD information of the future direction is decoded. In stepS1522, MVP information of the future direction is decoded. When, in stepS1519, the prediction direction is the past direction or thebi-direction, or when step S1522 is terminated, it is determined whetherthe decoding of all sub decoding blocks is finished in step S1523. Here,the process of this step is performed even after step S1513 is finished.When finished, the flowchart is terminated. When not finished, theprocess moves from the current sub decoding block to a next sub decodingblock in step S1524, and the above-described process is repeated fromstep S1501.

In this embodiment, a method of partitioning the current coding block bysearching for a motion boundary point at a boundary of the currentcoding block using motion information around the current coding block isillustrated. Further, a method of predicting each sub coding block usinga Decoder-side Motion Vector Derivation (DMVD) mode is illustrated.

FIG. 16 shows a block partitioning unit in a video encoding apparatusaccording to one embodiment. The higher depth block is subjected toprocesses 1601 to 1604 as the same processes as the processes 201 to 204in FIG. 2 to determine the current block. When MT (Motion Tree)partitioning (1605) a higher depth block, a motion boundary point issearched for (1606). Here, the MT partitioning refers to a method ofpartitioning a block at a motion boundary point by searching for themotion boundary point in the higher depth block. A method of searchingfor the motion boundary point will be described in detail below. Thecurrent block is determined by partitioning (1607) the higher depthblock based on the searched motion boundary point. The method forpartitioning the higher depth block is described in detail below.

FIG. 17 shows a block partitioning unit in the video decoding apparatusaccording to one embodiment. Partitioning information is extracted for ahigher depth block (1701). One of QT partitioning, BT partitioning, orMT partitioning is determined based on the partitioning information.Steps 1702 to 1705 as the same as steps 1302 to 1305 in FIG. 13 arecarried out to determine the current block. In the MT partitioning(1706), a motion boundary point is searched for (1707). A method ofsearching for a motion boundary point will be described in detail below.The current block is determined by partitioning (1708) a higher depthblock based on the searched motion boundary point. The method forpartitioning the higher depth block is described in detail below.

In FIG. 16 and FIG. 17 , QT, BT, and MT partitionings may be applied toa range from the maximum coding block to the minimum coding block.However, partitioning may be performed using only some partitioningmethods depending on a size and a depth of the coding block. Forexample, it is assumed that a size of the maximum coding block is 64x64,and a size of the minimum coding block is 4×4. FIG. 32 may be referredto for a case where the size of the coding block to which QTpartitioning is applicable is 64×64 to 16×16, the size of the maximumcoding block to which BT partitioning and MT partitioning are applicableis 16×16, and a partitionable depth is 3. In FIG. 32 , a solid linemeans QT partitioning, a dotted line means BT partitioning, and a dottedcombination line means MT partitioning. As shown in FIG. 32 ,partitioning of the coding block may be performed under theabove-described condition. When MT partitioning occurs, the currentcoding block may be partitioned into coding blocks of the odd-valuedsize. When the coding block having a size of 17×8 occurs, and BTpartitioning is performed on the coding block, the coding block may bepartitioned into 9×4 and 8×4 coding blocks. Further, the maximum size,the minimum size, and the partitionable depth of the coding block foreach partitioning method may be previously transmitted in a higherheader.

FIG. 18 shows a prediction unit in the video encoding apparatusaccording to one embodiment. Processes 1801 to1807 as the same processesas the processes 301 to 307 in FIG. 3 are performed. A prediction blockmay be generated using the optimal intra prediction mode determined viaprocesses 1801 to 1802. Alternatively, motion compensation 1810 may beperformed using the optimal motion information determined via processes1803 to 1807 to generate a prediction block. In addition, when interprediction is performed, there is a motion information determinationmethod using a DMVD mode. In order to perform motion estimation in thesame manner in the video encoding/decoding apparatus, initial motioninformation is determined (1808) using motion information of areconstructed region. The motion estimation is performed (1809) usingthe determined initial motion information to determine optimal motioninformation. Then, a prediction block is generated by performing motioncompensation (1810).

FIG. 19 shows a prediction unit in the video decoding apparatusaccording to one embodiment. Processes 1901 to1907 as the same processesas the processes 1401 to1407 in FIG. 14 are performed. A predictionblock may be generated using the optimal intra prediction modedetermined via processes 1901 to 1902. Alternatively, motioncompensation (1910) may be performed using the optimal motioninformation determined via processes 1903 to 1907 to generate aprediction block. In addition, when inter prediction is performed, thereis a motion information determination method using DMVD mode. In orderto perform motion estimation in the same manner in the videoencoding/decoding apparatus, initial motion information is determined(1908) using motion information of a reconstructed region. The motionestimation is performed (1909) using the determined initial motioninformation to determine optimal motion information. A prediction blockis generated by performing motion compensation (1910).

The optimal motion information obtained via motion estimation using atemplate block in the DMVD motion estimators 1809 and 1909 in FIG. 18and FIG. 19 may be applied to the current prediction block.Alternatively, the reconstructed motion information in the templateblock may be determined as the optimal motion information of the currentprediction block. Further, when the current coding block is generated byMT partitioning of a higher depth block, only the DMVD mode may be usedas the prediction mode of the current coding block. Further, optimalmotion information derived in the DMVD mode may be used as a candidatefor the AMVP mode.

FIG. 20 shows a DMVD motion estimator in the prediction unit in a videoencoding/decoding apparatus. After performing the DMVD mode using theinitial motion information determined by the DMVD initial motioninformation detector in the video encoding/decoding apparatus, optimalmotion information is determined. The DMVD mode includes a mode thatuses a template (hereinafter, referred to as ‘template matching mode’)and a mode that does not use a template (hereinafter, referred to as‘bi-directional matching mode’). When the bi-directional matching modeis used (2001), a uni-direction motion vector of each initial motioninformation is linearly scaled to a reference picture in an oppositeprediction direction. Here, the scaling of the motion vector isperformed in proportion to a distance between the current picture andthe reference picture in each direction. After determining the motionvector in bi-direction like this (2002), the motion vector of eachdirection that minimizes a difference between a prediction block of apast direction and a prediction block of a future direction isdetermined as optimal motion information (2003). A drawing 2102 in FIG.21 shows a method of linearly generating the motion vectors in the pastand future directions of the current block in the bi-directionalmatching mode and then generating the prediction block for the currentblock as an average of the two prediction blocks of the bi-direction.When the template matching mode is used (2004), a template block isdetermined in the reconstructed region around the current block (2005).The template block may be configured as shown in examples 2201 to 2203in FIG. 22 . In the drawing 2201, the template block is determined in abottom-left (Template A), a top-left (Template B), a top-left (TemplateC), and a top-right (Template D) of the current block, respectively. Thesize and shape of each template block may be determined variously. Inthe drawing 2202, the template block is determined in a bottom-left(Template A), a top-left (Template B), a top-left (Template C), and atop-right (Template D) of the current block, respectively. A differencetherebetween is in that both the left and above reconstructed regionsadjacent to the current block are used in the drawing 2202. The drawing2203 refers to a method of generating a template block by consideringthe template block generating methods in the drawings 2201 and 2202 atthe same time. In addition, the template block may be generated in thereconstructed region around the current block via various methodsincluding a method in which left and above reconstructed regionsadjacent to the current block is determined as a single template block.However, information indicating the shape and size of the template blockmay be transmitted from the video encoding apparatus. After performingmotion estimation (2006) which detects the prediction block most similarto each determined template block from the reference picture, theoptimal motion information for each template block is estimated, andthen motion information most suitable for the current block is detectedfrom the motion information. Thus, the detected motion information isdetermined as an optimal motion information. Here, when estimatingmotion information optimal for the template block, the motion estimationmay be performed based on an arbitrary pattern selected among fourmotion estimated patterns shown in FIG. 10 . The cost value of motionestimation means the sum of the amount of prediction error and theamount of a virtual bit of the motion information. The prediction errormay be obtained via various calculation methods such as SAD (Sum ofAbsolute Difference), SATD (Sum of Absolute hadamard TrnasformDifference), and SSD (Sum of Square Difference). Equations (3), (4), and(5) show the calculation methods of SAD, SATD, and SSD, respectively.

$\begin{matrix}{{{SAD} = {\sum\limits_{i,j}{❘{{Diff}( {i,j} )}❘}}},{{{Diff}( {i,j} )} = {{{Template}( {i,j} )} - {{PredBlk}( {i,j} )}}}} & \lbrack {{Equation}3} \rbrack\end{matrix}$ $\begin{matrix}{{{SATD} = {\sum\limits_{i,j}\frac{❘( {{DiffT}( {i,j} )} ❘}{2}}},{{{DiffT}( {i,j} )} = {{HT}( {{{Template}( {i,j} )} - {{PredBlk}( {i,j} )}} )}}} & \lbrack {{Equation}4} \rbrack\end{matrix}$ $\begin{matrix}{{{SSD} = {\sum\limits_{i,j}{{Diff}( {i,j} )}^{2}}},{{{Diff}( {i,j} )} = {{{Template}( {i,j} )} - {{PredBlk}( {i,j} )}}}} & \lbrack {{Equation}5} \rbrack\end{matrix}$

where i, j means a pixel position, the template (i, j) means a pixel ofthe template block, and PredBlk(i, j) means a pixel of the predictionblock. Here, the HT( )function of Equation 4 refers to a function valueobtained by Hadamard transforming a difference block between thetemplate block and the prediction block. The virtual bit amount of themotion information is not actually transmitted information, but isobtained by calculating the virtual bit amount of the motion informationthat is expected to be the same in the video encoding apparatus and thevideo decoding apparatus. For example, a difference vector size amountbetween a motion vector of initial motion information and a motionvector in motion information under current motion estimation may becalculated and determined as the virtual bit amount. Alternatively, thevirtual bit amount of the motion information may be calculated based onthe bit amount of the reference picture information. A drawing 2101 inFIG. 21 shows a method in which when each of the left and abovereconstructed region adjacent to the current block is used as a singletemplate block, the prediction block for the template block most similarto the template block is detected and then a block adjacent to thecorresponding template block is determined as the prediction block forthe current block.

FIG. 23 is a flowchart showing a method of searching for a motionboundary point according to one embodiment. This flowchart is performedin a process of motion boundary point detection (1606, 1707) by theblock partitioning unit in the video encoding/decoding apparatusaccording to one embodiment. Index information of an initial motionboundary point (hereinafter referred to as ‘MB Idx’) is initialized to−1, and the number of template blocks is initialized to 2^(NumOfMB)denotes the number of motion boundary points. In this embodiment, it isassumed that the number of motion boundary points is limited to at mosttwo. However, the number may be larger than 2. The method initializesall arguments to infinity in the CostBuf[N] buffer and initializes allarguments to −1 in the IdxBuf[N] buffer. Thereafter, the initial motioninformation list is determined (S2301). Refer to FIG. 25 for the methodof determining the initial motion information list. The index in theFIG. 25 table means the priority of initial motion information.Candidate motion information in AMVP mode, candidate motion informationin merge mode, motion information of the sub-blocks in the reconstructedregions in the top, left, top-left, top-right, and bottom-leftdirections around the current block, zero motion information, etc. maybe considered as initial motion information. In addition, various motioninformation candidates based on reconstructed information may be used.Thereafter, MB Idx is updated by increasing MB Idx by 1 (S2302). Themethod creates a template block in the reconstructed region based on thecurrent motion boundary point (S2303), and searches for optimal motioninformation of each template block (S2304). Refer to FIG. 28 and FIG. 29for detailed descriptions of S2303 and S2304 steps. FIG. 28 and FIG. 29show a method for determining the template block in the reconstructedregion based on the current motion boundary point and then derivingoptimal motion information for each template block. In FIG. 28 , onlysome reconstructed regions are generated, based on the motion boundarypoint, as the template block. The prediction block most similar to thetemplate A block is detected from the reference picture and theprediction block most similar to the template B block is detected fromthe reference picture. The template C block means a block as acombination of the template A and B blocks. The block that is mostsimilar to the template C block is detected from the reference picture.In FIG. 29 , unlike FIG. 28 , template A, B, and C blocks are determinedbased on motion boundary points within the reconstructed regions as theleft and top regions adjacent to the current block. The prediction blockmost similar to each template block is detected from the referencepicture. Thereafter, the partitioning cost value at the current motionboundary point is calculated using the cost value (SAD, SATD, SSD costvalue, etc.) for the optimal motion information of each template block.Here, the partitioning cost value may be determined using variousinformation. A first method of determining the partitioning cost valueis to determine the sum of the cost value of the template A block andthe cost value of the template B as the partitioning cost value. Here,the cost value of the template C block is not used. Refer to Equation 6for a second method of determining the partitioning cost value.

Partitioning cost value=[1−(cost value of template A block+cost value oftemplate B)/cost value of template C block]×100.0   [Equation 6]

(where cost value of template C block≥(cost value of template Ablock+cost value of template B))

Equation 6 refer to the calculation of a change percentage of the sum ofthe cost values of template A and B blocks relative to the cost value oftemplate C block.

The method determines whether the partitioning cost value at the currentmotion boundary point is smaller than one or more element values in theCostBuf buffer (S2306). When the current partitioning cost value issmaller than one or more element values in the Costbuf buffer, thecurrent partitioning cost value is stored in the CostBuf[N−1] buffer andthe current MB Idx is stored in the IdxBuf[N−1] buffer (S2307).Thereafter, the element values in the CostBuf buffer are sorted in anascending order (S2308). The method sorts the element values in theIdxBuf buffer in the same manner as the CostBuf buffer sorting order(S2309). For example, when N is 4, 4 elements of the CostBuf buffer arestored as {100, 200, 150, 180}, and 4 elements of the IdxBuf buffer arestored as {0, 3, 2, 8}, CostBuf is sorted in the order of {100, 150,180, 200}, and IdxBuf is sorted in the order of {0, 2, 8, 3} in the samesorting manner as that in the CostBuf buffer. When the currentpartitioning cost value is greater than any element in the Costbufbuffer, or when step S2309 is terminated, it is determined whether thecurrent MB Idx is a last search candidate partitioning boundary point(S2310). When the current motion boundary point is the last searchcandidate partitioning boundary point, the flowchart is terminated.Otherwise, the method returns to step S2302 in which MB Idx is updatedand the above-described process is repeated.

FIG. 24 illustrates a method for detecting motion boundary pointsaccording to one embodiment, and is directed to an accelerated algorithmof the flow chart shown in FIG. 23 . As in FIG. 23 , this flowchart maybe performed in the process of motion boundary point detection (1606,1707) by the block partitioning unit in the video encoding/decodingapparatus according to one embodiment, or may be performed in the DMVDmotion estimator of the prediction unit. The initial information is thesame as the initial information in FIG. 23 . The method of setting theinitial motion information list (S2401) is the same as the descriptionof step S2301 in FIG. 23 . The left and top regions adjacent to thecurrent block and, when necessary, non-adjacent top-left, top-right, andbottom-left partial regions to the current block in the reconstructedregion are determined as template blocks, respectively (S2402). The costvalues of all motion information within the motion estimation rangeusing the initial motion information for the template block arecalculated and stored on a line basis (S2403). Here, refer to FIG. 26for a method of storing the cost value on the line basis. FIG. 26 showsthe method of storing the cost value on a line basis in a regiondetermined as a template block in the reconstructed region. H-Cost₀ toH-Cost_(H−1) mean that the cost values corresponding to the motioninformation in a left region adjacent to the current block are stored ona line basis. W-Cost₀ to W-Cost_(W−1) mean that the cost valuescorresponding to the motion information in a top region adjacent to thecurrent block are stored on a line basis. Thereafter, the process ofupdating MB Idx (S2404) is the same as the description of step S2302 inFIG. 23 . Thereafter, the template block is recreated in thereconstructed region based on the current motion boundary point (S2405).The method of recreation of the template block is the same as thedescription of step S2303 in FIG. 23 . The cost value per line for theoptimal motion information of the recreated template block ispre-calculated in the S2403 step. Thus, the partitioning cost value atthe current motion boundary point is calculated using the correspondingcost value per line (S2406). Here, the method for calculating thepartitioning cost value is the same as that described in step S2305 inFIG. 3 . Steps S2407 to S2410 are the same as the descriptions of stepsS2306 to S2309 in FIG. 23 . When the current MB Idx is not the lastsearch candidate partitioning boundary point in step S2411, the methodreturns to step S2404 in which the above-described process is repeated.

The higher depth block may be partitioned at one or two motion boundarypoints based on FIG. 23 and FIG. 24 . Here, when a difference betweentwo higher partitioning cost values in the CostBuf buffer is greaterthan a certain threshold, the corresponding block may be partitioned atone motion boundary point. Otherwise, when the two higher partitioningcost values are similar to each other, the block may be partitioned attwo motion boundary points. Further, when transmitting the correspondingmotion boundary point information to the video decoding apparatus, theprocess of FIG. 23 and FIG. 24 may be omitted in the video decodingapparatus. The content for transmitting the motion boundary pointinformation is not included in the encoding/decoding information in thevideo encoding/decoding apparatus to be described later. The optimal MBIdx among all MB Idx may be directly transmitted. However, afterdetermining M candidate MB Idx, a motion boundary point at which theRD-cost value is optimal among M MB Idx may be transmitted.

FIG. 27 shows a method of partitioning a higher depth block (a currentblock) based on the number of motion boundary points. The MB Idx may beknown in an order in which the partitioning cost values are smallerbased on the process of FIGS. 23 and 24 . Thus, when only one motionboundary point is used, the motion boundary point corresponding to theMB Idx with the smallest partitioning cost value is determined as thepartitioning boundary point as in a drawing 2701. Thus, the currentblock is partitioned into lower depth blocks (prediction blocks) A andB. The reconstruction of the lower depth block A is finished, andoptimal motion information for the lower depth block B may be derived byperforming motion estimation again using the template A block and aregion under the lower depth block A. When two motion boundary pointsare used and there is at least one MB Idx in each of the top and leftdirections in the IdxBuf array of FIG. 23 and FIG. 24 , as shown in thedrawing 2702, a motion boundary point corresponding to MB Idx with thesmallest partitioning cost value in the top direction and a motionboundary point corresponding to MB Idx with the smallest partitioningcost value in the left direction are determined as partitioning boundarypoints. Then, the current block is partitioned into lower depth blocksA, B, C, and D. The reconstruction of the lower depth block A isfinished, and then optimal motion information for the lower depth blockB may be derived by performing motion estimation again using thetemplate D block and the right region adjacent to the lower depth blockA. The reconstruction of the lower depth block A is finished and optimalmotion information the lower depth block C may be derived by performingmotion estimation again using the template A block and the region underthe lower depth block A. The reconstruction of the lower depth blocks Band C is finished, and optimal motion information for the lower depthblock D may be derived by performing motion estimation again using theright region adjacent to the lower depth block C and the region underthe lower depth block B.

The prediction block A may be obtained via the above-described DMVDmotion estimation using each of the template B and C blocks as onetemplate block. The prediction block B may be obtained via theaforementioned DMVD motion estimation using the template A block as onetemplate block. However, when the prediction block A is firstreconstructed by performing transform/quantization anddequantization/inverse-transform processes not on a current block basisbut on a prediction block basis, the prediction block B may be obtainedvia DMVD motion estimation using a template C′ block and the template Ablock in the reconstructed region of the prediction block A as onetemplate block. When two motion boundary points are used and there is atleast one MB Idx in each of the top and left directions in the IdxBufarray of FIG. 23 and FIG. 24 , as shown in the drawing 2702, two motionboundary points corresponding to MB Idx with the smallest partitioningcost value in the top direction and MB Idx with the smallestpartitioning cost value in the left direction are determined aspartitioning boundary points. The current block is partitioned intolower depth blocks A, B, C, and D.

The block A may be obtained via the aforementioned DMVD motionestimation using each of the template B and C blocks as one templateblock. The prediction block B may be obtained via the above-describedDMVD motion estimation using the template block D as one template block.The prediction block C may be obtained via the above-described DMVDmotion estimation using the template block A as one template block. Theprediction block D may be obtained via the above-described DMVD motionestimation using prediction blocks D and A as one template block.However, when the transform/quantization and inversequantization/inverse transform processes are performed not on a currentblock basis but on a prediction block basis, the prediction block A maybe reconstructed and then the prediction block B may be generated andreconstructed using the template B′ block and the template D block asone template block, and then the prediction block C may be generated andreconstructed using the template A block and template C′ block as onetemplate block, and, then, the prediction block D may be generated andreconstructed using the template D′ block and the template A′ block asone template block.

FIG. 30 is a flow chart showing a flow of encoding some codinginformation in the entropy encoding unit in a video encoding apparatus.In this embodiment, an exemplary method of encoding partitioninginformation and prediction information of a block will be described.First, information indicating whether the coding block is partitioned isencoded (S3001). It is determined whether the coding block ispartitioned (S3002). When it is determined that the coding block ispartitioned, information indicating which one of QT partitioning, BTpartitioning, and MT partitioning is performed is encoded (S3003). Here,when MT partitioning is performed by using fixedly one or two motionboundary points, there is no additional encoding information. However,when one or two motion boundary points are adaptively used according tothe characteristics of the current block, information on how many motionboundary points are used is additionally encoded. Further, when apartitioning range is out of a preset partitioning range for eachpartitioning method, one of the remaining partitioning method except forthe corresponding partitioning method for which the partitioning rangeis out of the preset partitioning range is encoded. Thereafter, thecurrent coding block is partitioned (S3004) according to thepartitioning method at the partitioning boundary point, and the processthen moves to a partitioned first encoding target sub-block (S3005).Thereafter, the process returns to step S3001 in which theabove-described process is repeated. When it is determined whether thecoding block is not partitioned (S3002), it is determined whether thecurrent coding block is a block partitioned based on the MT partitioningmethod (S3006). When the current coding block is not a block partitionedbased on the MT partitioning method, the information about activation ofthe skip mode is encoded (S3007). It is determined whether the skip modeis activated (S3008). When the skip mode is not activated, predictionmode information is encoded (S3009). The type of the prediction mode isdetermined (S3010). When the corresponding prediction mode is not aninter prediction mode, intra prediction information is encoded (S3011)and then the process moves to step S3030. Subsequent steps will bedescribed in detail below. When it is determined that the skip mode isactivated in step S3008, the DMVD mode operation information is encoded(S3012). It is determined whether the DMVD mode is activated (S3013).When the DMVD mode is activated, template information is encoded(S3014). Here, the template information refers to information indicatingwhich template block among the template blocks in the reconstructedregion whose motion information is used to predict the current block.When there is one template block, the corresponding information may notbe encoded. Thereafter, the process moves to step S3030. Subsequentsteps will be described in detail below. When the current coding blockis a block partitioned based on the MT partitioning method in stepS3006, the template information is encoded (S3019). Then, the processmoves to step S3030. Subsequent steps will be described in detail belowas well. When the DMVD mode is not activated in step S3013, the mergecandidate index information for the skip mode is encoded (S3020). When,in step S3010, the prediction mode is an inter prediction mode,information about activation of the merge mode is encoded (S3015). It isdetermined whether the merge mode is activated (S3016). When the mergemode is activated, information about activation of the DMVD mode isencoded (S3017). It is determined whether the DMVD mode is activated(S3018). When the DMVD mode is activated, the template information isencoded (S3019). When the DMVD mode is not activated, the mergecandidate index information for the merge mode is encoded (S3020). Aftersteps S3019 and S3020 are terminated, the process moves to step S3030.Subsequent steps will be described in detail below as well. When themerge mode is not activated in step S3016, the prediction direction isencoded (S3021). The prediction direction may mean one of a pastdirection, a future direction, and a bi-direction. It is determinedwhether the prediction direction is the future direction (S3022). Whenthe prediction direction is not the future direction, the referencepicture index information of the past direction, MVD information of thepast direction, and MVP information of the past direction are encoded(S3023, S3024, S3025). When the prediction direction is the futuredirection or after step S3025 is finished, it is determined whether theprediction direction is the past direction (S3026). When the predictiondirection was not the past direction, the reference picture indexinformation of the future direction, MVD information of the futuredirection, and MVP information of the future direction are encoded(S3027, S3028, S3029). When the prediction direction is the pastdirection or after step S3029 is finished, the process moves to stepS3030. In the corresponding step S3030, it is determined whetherencoding of all sub coding blocks is ended (S3030). When encoding of allsub coding blocks is ended, this flow chart is terminated. Otherwise,the process moves to a next sub coding block (S3031), and then theprocess returns to step S3001 in which the above-described process isrepeated.

FIG. 31 is a flowchart showing a flow of decoding some codinginformation in the entropy decoding unit in a video decoding apparatus.In this embodiment, a method of decoding encoding information of somecoding information in the entropy encoding unit in the video encodingapparatus of FIG. 30 will be described. First, information indicatingwhether the decoding block is partitioned is decoded (S3101). It isdetermined whether or not the decoding block is partitioned (S3102).When the decoding block is partitioned, information indicating whichpartitioning method among QT partitioning, BT partitioning, and MTpartitioning is performed is decoded (S3103). Here, when MT partitioningis performed by using fixedly one or two motion boundary points, thereis no additional decoding information. However, when one or two motionboundary points are adaptively used according to the characteristics ofthe current block, information on how many motion boundary points areused is additionally decoded. Thereafter, the current decoding block ispartitioned (S3104) according to the partitioning method at thepartitioning boundary point, and the process then moves to a partitionedfirst sub decoding block (S3105). Thereafter, the process returns tostep S3101 in which the above-described process is repeated. When it isdetermined that the decoding block is not partitioned in S3102, it isdetermined whether the current decoding block is a block partitionedbased on the MT partitioning method (S3106). When the current decodingblock is not a block partitioned based on the MT partitioning method,the information about activation of the skip mode is decoded (S3107). Itis determined whether the skip mode is activated (S3108). When the skipmode is not activated, prediction mode information is decoded (S3109).The type of the prediction mode is determined (S3110). When thecorresponding prediction mode is not an inter prediction mode, intraprediction information is decoded (S3111) and then the process moves tostep S3130. Subsequent steps will be described in detail below. When itis determined that the skip mode is activated in step S3108, informationabout activation of the DMVD mode is decoded (S3112). It is determinedwhether the DMVD mode is activated (S3113). When the DMVD mode isactivated, template information is decoded (S3114). Thereafter, theprocess moves to step S3130. Subsequent steps will be described indetail below. When the current decoding block is a block partitionedbased on the MT partitioning method in step S3106, the templateinformation is decoded (S3119). Then, the process moves to step S3130.Subsequent steps will be described in detail below as well. When theDMVD mode is not activated in step S3113, the merge candidate indexinformation for the skip mode is decoded (S3120). When, in step S3110,the prediction mode is an inter prediction mode, information aboutactivation of the merge mode is decoded (S3115). It is determinedwhether the merge mode is activated (S3116). When the merge mode isactivated, information about activation of the DMVD mode is decoded(S3117). It is determined whether the DMVD mode is activated (S3118).When the DMVD mode is activated, the template information is decoded(S3119). When the DMVD mode is not activated, the merge candidate indexinformation for the merge mode is decoded (S3120). After steps S3119 andS3120 are terminated, the process moves to step S3130. Subsequent stepswill be described in detail below as well. When the merge mode is notactivated in step S3116, the prediction direction is decoded (S3121).The prediction direction may mean one of a past direction, a futuredirection, and a bi-direction. It is determined whether the predictiondirection is the future direction (S3122). When the prediction directionis not the future direction, the reference picture index information ofthe past direction, MVD information of the past direction, and MVPinformation of the past direction are decoded (S3123, S3124, S3125).When the prediction direction is the future direction or after stepS3125 is finished, it is determined whether the prediction direction isthe past direction (S3126). When the prediction direction was not thepast direction, the reference picture index information of the futhredirection, MVD information of the futhre direction, and MVP informationof the futhre direction are decoded (S3127, S3128, S3129). When theprediction direction is the past direction or after step S3129 isfinished, the process moves to step S3130. In the corresponding stepS3130, it is determined whether decoding of all sub decoding blocks isended (S3130). When decoding of all sub decoding blocks is ended, thisflow chart is terminated. Otherwise, the process moves to a next subdecoding block (S3131), and then the process returns to step S3101 inwhich the above-described process is repeated.

FIG. 33 shows a DMVD motion estimator in the prediction unit in a videoencoding/decoding apparatus. After performing the DMVD mode using theinitial motion information determined by the DMVD initial motioninformation detector in the video encoding/decoding apparatus, optimalmotion information is determined. The DMVD mode includes a mode thatuses a template (hereinafter, referred to as ‘template matching mode’)and a mode that does not use a template (hereinafter, referred to as‘bi-directional matching mode’). When the bi-directional matching modeis used (3301), an uni-direction motion vector of each initial motioninformation is linearly scaled to a reference picture in an oppositeprediction direction. Here, the scaling of the motion vector isperformed in proportion to a distance between the current picture andthe reference picture in each direction. After determining the motionvector in the bi-direction (3302), the motion vector of each directionthat minimizes a difference between a prediction block of the pastdirection and a prediction block of the futhre direction is determinedas optimal motion information (3303). A drawing 2102 in FIG. 21 shows amethod of linearly generating the motion vectors in the past and futuredirections of the current block in the bi-directional matching mode andthen generating the prediction block for the current block as an averageof the two bi-directional prediction blocks. When the template matchingmode is used (3304), the number of template blocks in the reconstructedregion is determined. When a single template block is used (3305)(hereinafter referred to as a ‘single template matching mode’), the leftand top reconstructed regions adjacent to the current block aredetermined as a template block, and optimal motion information isdetermined via motion estimation 3306 using the corresponding templateblock. The drawing 2101 in FIG. 21 shows a method for searching for theprediction block of the template block most similar to the templateblock in the single template matching mode, and then determining a blockadjacent to the corresponding template block as the prediction block forthe current block. Here, when estimating motion information optimal forthe template block, the motion estimation may be performed based on anarbitrary pattern selected from the four motion estimated patterns shownin FIG. 10 . The cost value of motion estimation means the sum of theamount of prediction error and the amount of a virtual bit of the motioninformation. The prediction error may be obtained via variouscalculation methods such as SAD (Sum of Absolute Difference), SATD (Sumof Absolute hadamard Trnasform Difference), and SSD (Sum of SquareDifference). This is the same as described with reference to FIG. 20 .Thus, detailed description thereof will be omitted.

When a plurality of template blocks are used (3307) (hereinafterreferred to as a ‘plural-templates matching mode’), whether a pluralityof template blocks in the reconstructed region are to be used in a fixedmanner, or the template block is adaptively generated using motionboundary points is determined. When the former is employed (3308), thetemplate block may be configured as shown in the example drawings 2201to 2203 in FIG. 22 . In the drawing 2201, the template blocks aredetermined as a bottom-left (Template A), a top-left (Template B), atop-left (Template C), and a top-right (Template D) adjacent to thecurrent block, respectively. The size and shape of each template blockmay be determined variously. In a similar manner to the drawing 2201, inthe drawing 2202, the template blocks are determined as a bottom-left(Template A), a top-left (Template B), a top-left (Template C), and atop-right (Template D) adjacent to the current block, respectively. Adifference therebetween is in that both of the left and abovereconstructed regions adjacent to the current block are used in thedrawing 2202. The drawing 2203 refers to a method of generating atemplate block by considering the template block generating methods inthe drawings 2201 and 2202 at the same time. In addition, the templateblock may be generated in the reconstructed region around the currentblock via various methods including a method in which left and abovereconstructed regions adjacent to the current block are determined as asingle template block. However, information indicating the shape andsize of the template block may be transmitted from the video encodingapparatus. After performing motion estimation (3306) which detects theprediction block most similar to each determined template block from thereference picture, the optimal motion information for each templateblock is estimated, and then motion information most suitable for thecurrent block is detected from the estimated motion information. Thus,the detected motion information is determined as an optimal motioninformation. When the template block is generated using motion boundarypoints rather than a a fixed template block, the motion boundary pointis first detected (3309). The motion boundary point detection methodwill be described in detail below. Motion information optimal for eachof template blocks is estimated via motion estimation for each oftemplate blocks divided based on the determined motion boundary point(3310). Optimal motion information for an adjacent template block isapplied for each of the prediction blocks divided based on the motionboundary point. Further, in the plural-templates matching mode, optimalmotion information obtained via motion estimation using the templateblock may be applied to the current prediction block. Alternatively, thereconstructed motion information in the template block may be directlydetermined as the optimal motion information for the current predictionblock.

FIG. 34 is a flowchart illustrating a method of determining a slope of amotion boundary line that partitions the current block at the determinedmotion boundary point. The first initial information initializes motionboundary line slope index information (hereinafter referred to as ‘MBAngle’) to −1, and initializes the optimal partitioning cost value(hereinafter referred to as ‘PartCost_(Best)’) to infinity. The MB Angleis updated by increasing a current MB Angle by 1 (S3401). Here, the MBAngle may be pre-set in the same manner in the video encoding/decodingapparatus using various methods. Thereafter, the motion boundary searchregion is partitioned into two template blocks by partitioning themotion boundary search region along the motion boundary line slopedirection corresponding to the current MB Angle. Then, the methodcalculates a cost value for optimal motion information via DMVD motionestimation for each template block (S3402). The partitioning cost valuefor the current MB Angle (hereinafter referred to as‘PartCost_(MB Angle)’) is calculated using the cost value for theoptimal motion information for each template block. The method ofcalculating the PartCost_(MB Angle) is the same as the method in FIG. 23described above. Refer to FIG. 35 for detailed descriptions of stepsS3402 and S3403.

FIG. 35 is a drawing showing a method of determining the template blockin the motion boundary search region along the slope direction of themotion boundary line based on the current motion boundary point, andderiving optimal motion information for each template block. The methoddetects the prediction block most similar to the template A block fromthe reference picture and detects the prediction block most similar tothe template B block from the reference picture. The template C blockmeans a block as a combination of the template A and B blocks. Themethod detects the block that is most similar to the template C blockfrom the reference picture. A line extending across the current block inthe motion boundary line slope direction is referred to as a virtualmotion boundary line. In this case, an actual motion boundary lineregarding pixels on and along which the virtual motion boundary lineextends may be determined based on which one of two prediction blocksaround the virtual line contains more pixel regions. The methoddetermines whether the PartCostMB Angle is smaller than the PartCostBestin the current MB Angle (S3404). When the partCostMB Angle is smallerthan PartCostBest in the current MB Angle, the PartCostMB Angle isstored in PartCostBest. The method stores the current MB Angle in theoptimal motion boundary line slope index information (S3405). When thepartCostMB Angle is greater than or equal to PartCostBest in the currentMB Angle or after step S3405 is finished, it is determined whether thecurrent MB Angle is the last search candidate motion boundary line slopeindex (S3406) When the current MB Angle was the last search candidatemotion boundary line slope index, this flowchart is terminated. Whenthis is not that case, the method returns to step S3401 in which theabove-described process is repeated.

When the reconstructed process is not first performed on each predictionblock basis as divided based on the motion boundary point, the methodmay create prediction blocks for the current block, and then may performa filtering process at the prediction block boundary.

In FIG. 36 , the filtering method is described using an example in whichthere is one motion boundary point and partitioning is performed alongthe motion boundary line which has a straight horizontal line directionrather than an oblique line direction. A filtering method at theprediction block boundary may vary. For example, the filtering may beperformed using a weighted sum of adjacent pixels of each predictionblock in a filtering region. Equation 7 is a filtering Equation.

a′=(W1×a)+(W2×b) b′=(W3×a)+(W4×b)   [Equation 7]

In Equation 7, a′ and b′ are filtered values of a and b predictedpixels. W1 to W4 are weight coefficients, and W1+W2=1, and W3+W4=1. Inthis case, a and b predicted pixels may be filtered by substituting 0.8for W1 and W4 and 0.2 for W2 and W3. In addition, various filteringmethods may be used.

In FIG. 37 , the filtering method is described using an example in whichthere is one motion boundary point and partitioning is performed alongthe motion boundary line which has an oblique line direction. In adrawing 3701, filtering may be performed using the weighted sum based onEquation 7 of pixels of prediction blocks adjacent to the motionboundary line in a manner similar to the description of FIG. 36 . In thefiltering method of a drawing 3702, when a prediction block is dividedbased on a virtual motion boundary line, all pixels passing through thevirtual motion boundary line are included in each of partitioned blocks(partitioned blocks A and B). Thereafter, a prediction block is createdfor each partitioned block. The filtering may be performed not based onthe actual motion boundary line but based on the virtual motion boundaryline. For each of pixels passing through the corresponding boundaryline, which prediction block contains a larger number of pixel regionsis calculated. In this drawing, pixel region areas p and q arecalculated based on a line through which the virtual motion boundaryline passes in a pixel S to be filtered. A pixel S is filtered usingEquation 8.

$\begin{matrix}{S^{\prime} = {( {\frac{p}{p + q} \times a} ) + ( {\frac{q}{p + q} \times b} )}} & \lbrack {{Equation}8} \rbrack\end{matrix}$

S′ is a filtered S pixel, a pixel a is a predicted pixel of apartitioned block A, and a b pixel is a predicted pixel of a partitionedblock B. Thus, the filtering may be carried out as in the example of thedrawings 3701 and 3702. In addition, various filtering methods may beused.

FIG. 38 is a flow chart showing a flow of encoding predictioninformation in the entropy encoding unit in a video encoding apparatus.First, the information about activation of the skip mode is encoded(S3801). It is determined whether the skip mode is activated (S3802).When the skip mode is activated, the prediction mode is encoded (S3803).It is determined whether the prediction mode is an inter prediction mode(S3804). When the prediction mode is not the inter prediction mode,intra prediction information is encoded and the flow chart isterminated. When the skip mode is activated in step S3802, informationabout activation of the DMVD mode is encoded (S3806). Here, theinformation about activation of the DMVD mode indicates whether the DMVDmode is activated (bi-directional matching mode or template matchingmode) or not. For example, when the DMVD mode is not activated, theinformation is encoded into 0. When the DMVD mode is the bi-directionalmatching mode, the information is encoded into 10. When DMVD mode is thetemplate matching mode, the information is encoded into 11. It isdetermined whether the DMVD mode is activated (S3807). When the DMVDmode is activated (that is, bi-directional matching mode or templatematching mode), template information is encoded (S3808). Here, templateinformation is encoded only when the template matching mode is theplural-templates matching mode. In the plural-templates matching mode,when a fixed template block is used, the template information indicateswhich template block whose motion information is used to create theprediction block. When using the motion boundary point, the templateinformation indicates which template block among the template blockspartitioned based on the motion boundary point whose motion informationis used to create the current prediction block. The template informationis encoded and then the flow chart is terminated. When the DMVD mode isnot activated, the merge candidate index information for the skip modeis encoded (S3809), and then this flowchart is terminated. When theprediction mode is the inter prediction mode in step S3804, informationabout activation of the merge mode is encoded (S3810). It is determinedwhether the merge mode is activated (S3811). When the merge mode is notactivated, the prediction direction is encoded (S3812). The predictiondirection may mean one of a past direction, a future direction, and abi-direction. It is determined whether the prediction direction is thefuture direction (S3813). When the prediction direction was not thefuture direction, the past reference picture index information, the pastMVD information, and the past MVP information are encoded (S3814, S3815,S3816). When the prediction direction is the future direction or afterstep S3816 is finished, it is determined whether the predictiondirection is the past direction (S3817). When the prediction directionis not the past direction, the future reference picture indexinformation, the future MVD information, and the future MVP informationare encoded (S3818, S3819, S3820). Then, the flow chart is terminated.This flowchart is terminated when the prediction direction is the pastdirection in step S3817. When the merge mode is activated in step S3811,information about activation of the DMVD mode is encoded (S3821). Thedescription thereof is the same as that of step S3806. It is determinedwhether the DMVD mode is activated (S3822). When the DMVD mode isactivated, the merge candidate index information for the merge mode isencoded and then this flowchart is terminated. When the DMVD mode is notactivated in step S3822, the template information is encoded (S3823) andthen the flow chart ends.

FIG. 39 is a flow chart showing a flow of decoding predictioninformation in the entropy decoding unit in a video decoding apparatus.First, the information about activation of the skip mode is decoded(S3901). It is determined whether the skip mode is activated (S3902).When the skip mode is activated, the prediction mode is decoded (S3903).It is determined whether the prediction mode is an inter prediction mode(S3904). When the prediction mode is not the inter prediction mode,intra prediction information is decoded and the flow chart isterminated. When the skip mode is activated in step S3902, informationabout activation of the DMVD mode is decoded (S3906). Here, theinformation about activation of the DMVD mode indicates whether the DMVDmode is activated (bi-directional matching mode or template matchingmode) or not. For example, when the DMVD mode is not activated, theinformation is decoded into 0. When the DMVD mode is the bi-directionalmatching mode, the information is decoded into 10. When DMVD mode is thetemplate matching mode, the information is decoded into 11. It isdetermined whether the DMVD mode is activated (S3907). When the DMVDmode is activated (that is, bi-directional matching mode or templatematching mode), template information is decoded (S3908) and theflowchart is ended. When the DMVD mode is not activated, the mergecandidate index information for the skip mode is decoded (S3909), andthen this flowchart is terminated. When the prediction mode is the interprediction mode in step S3904, information about activation of the mergemode is decoded (S3910). It is determined whether the merge mode isactivated (S3911). When the merge mode is not activated, the predictiondirection is decoded (S3912). The prediction direction may mean one of apast direction, a future direction, and a bi-direction. It is determinedwhether the prediction direction is the future direction (S3913). Whenthe prediction direction was not the future direction, the referencepicture index information of the past direction, MVD information of thepast direction, and MVP information of the past direction are decoded(S3914, S3915, S3916). When the prediction direction is the futuredirection or after step S3916 is finished, it is determined whether theprediction direction is the past direction (S3917). When the predictiondirection is not the past direction, the reference picture indexinformation of the future direction, MVD information of the futuredirection, and MVP information of the future direction are decoded(S3918, S3919, S3920). Then, the flow chart is terminated. Thisflowchart is terminated when the prediction direction is the pastdirection in step S3917. When the merge mode is activated in step S3911,information about activation of the DMVD mode is decoded (S3921). Thedescription thereof is the same as that of step S3906. It is determinedwhether the DMVD mode is activated (S3922). When the DMVD mode isactivated, the merge candidate index information for the merge mode isdecoded and then this flowchart is terminated. When the DMVD mode is notactivated in step S3922, the template information is decoded (S3923) andthen the flow chart ends.

The initial motion information may be created based on informationsignaled from the encoding apparatus. In the template matching mode, oneinitial motion information may be generated, whereas in thebi-directional matching mode, two initial motion information may begenerated. The improvement process may determine delta motioninformation that minimizes a difference between pixel values of the L0reference block and the L1 reference block for the current block, andmay improve the initial motion information based on the determined deltamotion information. The method of determining the delta motioninformation will be described detail with reference to FIG. 43 .

The motion compensation may be performed using the improved motioninformation (MV_(ref)) of the current block.

The improved motion information (MV_(ref)) of the current block may bestored in the buffer of the decoder, and may be used as a motioninformation predictor of a block (hereinafter referred to as a firstblock) to be decoded after the current block. The current block may be aneighboring block spatially/temporally adjacent to the first block. Thecurrent block may not be spatially/temporal adjacent to the first block,but may belong to the same CTU, tile, or tile group as the first block.

Alternatively, the improved motion information (MV_(ref)) of the currentblock may be used only for motion compensation for the current block,but may not be stored in the decoder's buffer. That is, the improvedmotion information (MV_(ref)) of the current block may not be used asthe motion information predictor of the first block.

Alternatively, the initial motion information (MV_(ref)) may be used formotion compensation for the current block. The improved motioninformation (MV_(ref)) may be stored in a buffer of the decoder and usedas a motion information predictor of the first block.

A pre-determined in-loop filter may be applied to the motion-compensatedcurrent block. The in-loop filter may include at least one of adeblocking filter, a sample adaptive offset (SAO), or an adaptive loopfilter (ALF). The improved motion information (MVref) may be used todetermine the attribute of the in-loop filter. Here, the attribute maymean boundary strength (bs), filtering strength, filter coefficient, thenumber of filter taps, filter type, and the like.

FIG. 40 shows a method of performing intra prediction in the predictionunit of a video encoding/decoding apparatus in an embodiment to whichthe present disclosure is applied.

Referring to FIG. 40 , the method may determine the intra predictionmode of the current block (S4000).

The current block may be divided into a luma block and a chroma block.The intra prediction mode pre-defined in the decoding apparatus mayinclude a non-directional mode (Planar mode, DC mode) and N directionalmodes. Here, a value of N may be an integer of 33, 65 or larger. Theintra prediction mode of the current block is determined for each of theluma block and the chroma block. This will be described below indetails.

The intra prediction mode of the luma block may be derived based on acandidate list (MPM list) and a candidate index (MPM Idx). The candidatelist includes a plurality of candidate modes. The candidate mode may bedetermined based on an intra prediction mode of a neighboring block tothe current block.

The number of candidate modes is k. k may be an integer of 3, 4, 5, 6,or greater. The number of candidate modes included in the candidate listmay be a fixed number pre-defined in the encoding/decoding apparatus ormay vary based on the attribute of the block. Here, the attribute of theblock may include a size, width (w) or height (h), ratio between widthand height, area, shape, partitioning type, partitioning depth, scanorder, encoding/decoding order, availability of the intra predictionmode, etc. The block may mean at least one of the current block orneighboring block thereto. Alternatively, the encoding apparatusdetermines the number of optimal candidate modes and then may encode thedetermined number into information and signal the information to thedecoding apparatus. The decoding apparatus may determine the number ofcandidate modes based on the signaled information. In this case, theinformation may indicate the maximum/minimum number of candidate modesincluded in the candidate list.

Specifically, the candidate list may include at least one of intraprediction mode of a neighboring block (modeN), modeN−n, or modeN+n, ora default mode.

For example, the neighboring block may include at least one of left (L),top (T), bottom-left (BL), top-left (TL), or top-right (TR) neighboringblock adjacent to the current block. Here, a value of n may be aninteger of 1, 2, 3, 4 or greater.

The default mode may include at least one of a planar mode, a DC mode,or a pre-determined directional mode. The pre-determined directionalmode may include at least one of a vertical mode (modeV), or ahorizontal mode (modeH), modeV−k, modeV+k, modeH−k, or modeH+k. Here, avalue of k may be greater than or equal to 1 or may be any value of anatural number smaller than or equal to 15.

The candidate index may specify a candidate mode identical with theintra prediction mode of the luma block among candidate modes of thecandidate list. That is, the candidate mode specified by the candidateindex may be set as the intra prediction mode of the luma block.

Alternatively, the intra prediction mode of the luma block may bederived by applying a pre-determined offset to the candidate modespecified by the candidate index. Here, the offset may be determinedbased on at least one of the size or shape of the luma block or a valueof the candidate mode specified by the candidate index. The offset maybe determined as an integer of 0, 1, 2, 3 or greater, or may bedetermined as a total number of intra prediction modes or directionalmodes pre-defined in the decoding apparatus. The intra prediction modeof the luma block may be derived by adding or subtracting the offset toor from the candidate mode specified by the candidate index. Theapplication of the offset may be selectively performed in considerationof the attribute of the block as aforementioned.

The intra prediction mode of the chroma block may be derived as shown inTable 1 or 2 below, based on information (intra_chroma_pred_mode)signaled from the encoding apparatus.

TABLE 1 lntraPredModeY[ xCb + cbWidth / 2 ][ yCb + cbHeight / 2 ]intra_chroma_pred_mode[ xCb ] X ( 0 <= [ yCb ] 0 50 18 1 X <= 66 ) 0 660 0 0 0 1 50 66 50 50 50 2 18 18 66 18 18 3 1 1 1 66 1 4 0 50 18 1 X

According to Table 1, the intra prediction mode of the chroma block maybe determined based on the signaled information, the intra predictionmode of the luma block, and a table pre-defined in the decodingapparatus. In Table 1, the mode66 means a diagonal mode in the top-rightdirection. The mode50 means a vertical mode. The mode18 means ahorizontal mode. The model may mean a DC mode. For example, when a valueof the signaled information intra_chroma_pred_mode is 4, the intraprediction mode of the chroma block may be set to be identical with theintra prediction mode of the luma block.

TABLE 2 IntraPredModeY[ xCb + cbWidth / 2 ][ yCb + cbHeight / 2 ]intra_chroma_pred_mode[ xCb ] X ( 0 <= [ yCb ] 0 50 18 1 X <= 66 ) 0 660 0 0 0 1 50 66 50 50 50 2 18 18 66 18 18 3 1 1 1 66 1 4 81 81 81 81 815 82 82 82 82 82 6 83 83 83 83 83 7 0 50 18 1 X

Table 2 may be applied when inter-components reference-based predictionfor the chroma block is allowed. In this case, the intra prediction modeis derived in the same way as in FIG. Table 1. Duplicate descriptionthereof will be omitted. However, Table 2 supports mode81, mode82, andmode83 as the intra prediction mode of the chroma block, and they areinter-components reference-based prediction modes.

Referring to FIG. 40 , the current block may be reconstructed based onat least one of the intra prediction mode derived in S4000 or aneighboring sample (S4010).

The reconstruction may be performed in units of sub-blocks of thecurrent block. To this end, the current block may be partitioned into aplurality of sub-blocks. Refer to FIG. 41 for details of thepartitioning method of the current block into the sub-blocks.

For example, the sub-blocks belonging to the current block may share thesingle intra prediction mode as derived. In this case, differentneighboring samples may be used for each of sub-blocks. Alternatively,the sub-blocks belonging to the current block may share a singlecandidate list as derived. A candidate index may be independentlydetermined for each sub-block.

In one example, when the derived intra prediction mode corresponds tothe inter-components reference-based prediction mode, the chroma blockmay be predicted from a previously-reconstructed luma block. This willbe described in details with reference to FIG. 42 .

FIG. 41 shows an intra prediction method on a sub-block basis in anembodiment to which the present disclosure is applied.

As described above, the current block may be partitioned into aplurality of sub-blocks. Here, the partitioning may be executed based onat least one of a quad tree (QT), a binary tree (BT), or a ternary tree(TT) as partitioning types as pre-defined in the decoding apparatus.Alternatively, the current block may correspond to a leaf node. The leafnode may mean a coding block that is no longer partitioned into asmaller coding block. That is, the partitioning of the current blockinto the sub-blocks may mean partitioning that is additionally performedafter partitioning is finally performed based on a partitioning typepre-defined in the decoding apparatus.

Referring to FIG. 41 , the partitioning may be performed based on a sizeof the current block (Embodiment 1).

Specifically, when the size of the current block 4100 is smaller than apre-determined threshold size, the current block may be partitioned intop sub-blocks in a vertical or horizontal direction. Conversely, when thesize of the current block 4110 is greater than or equal to the thresholdsize, the current block may be partitioned into q sub-blocks in avertical or horizontal direction or may be partitioned into q sub-blocksin vertical and horizontal directions. Here, p may be an integer of 1,2, 3 or greater. q may be 2, 3, 4 or greater. However, p may be set tobe smaller than q.

The threshold size may be signaled from the encoding apparatus or may bea fixed value pre-defined in the decoding apparatus. For example, thethreshold size may be expressed as N×M, where each of N and M may be 4,8, 16 or greater. N and M may be equal to each other or may be differentfrom each other.

Alternatively, when the size of the current block is smaller than thepredefined threshold size, the current block may not be partitioned(non-split). Otherwise, the current block may be partitioned into 2, 4,or 8 sub-blocks.

In another example, the partitioning may be performed based on a shapeof the current block (Embodiment 2).

Specifically, when the shape of the current block is square, the currentblock may be partitioned into 4 sub-blocks. Otherwise, the current blockmay be partitioned into 2 sub-blocks. Conversely, when the shape of thecurrent block is square, the current block may be partitioned into 2sub-blocks. Otherwise, the current block the current block may bepartitioned into 4 sub-blocks.

Alternatively, when the shape of the current block is square, thecurrent block may be partitioned into 2, 4, or 8 sub-blocks. Otherwise,the current block may not be partitioned. Conversely, when the shape ofthe current block is square, the current block may not be partitioned.Otherwise, the current block may be partitioned into 2, 4, or 8sub-blocks.

One of the above-described Embodiments 1 and 2 may be selectivelyapplied. Alternatively, the partitioning may be performed based on acombination of the Embodiments 1 and 2.

When partitioned into 2 sub-blocks, the current block may be partitionedinto 2 sub-blocks in either the vertical or horizontal direction. Whenpartitioned into 4 sub-blocks, the current block may be partitioned into4 sub-blocks in either the vertical or horizontal direction or may bepartitioned into 4 sub-blocks in the vertical and horizontal directions.When partitioned into 8 sub-blocks, the current block may be partitionedinto 8 sub-blocks in the vertical or horizontal direction or may bepartitioned into 8 sub-blocks in the vertical and horizontal directions.

The above embodiments exemplify the case in which the current block ispartitioned into 2, 4 or 8 sub-blocks. However, the present disclosureis not limited thereto. The current block may be partitioned into 3sub-blocks in either a vertical or horizontal direction. In this case,the ratio between the widths or the heights of the 3 sub-blocks may be1:1:2, 1:2:1, or 2:1:1.

Partitioning information about whether the current block is partitionedinto sub-blocks, whether the current block is partitioned into 4sub-blocks, the partitioning direction, the partitioning number may besignaled from the encoding apparatus and may be variably determinedbased on a pre-determined coding parameter by the decoding apparatus.Here, the coding parameter may include a block size/shape, apartitioning type (4-partitioning, 2-partitioning, 3-partitioning), theintra prediction mode, a range/location of a neighboring pixel for intraprediction, a component type (e.g., luma, chroma), a maximum/minimumsize of the transform block, a transform type (e.g., transform skip,DCT2, DST7, DCT8), and the like. Based on the partitioning information,the current block may be partitioned into a plurality of sub-blocks ornon-partitioned.

FIG. 42 shows an inter-components reference-based prediction method inan embodiment to which the present disclosure is applied.

The current block may be divided into a luma block and a chroma blockaccording to a component type. The chroma block may be predicted usingthe pixel of the reconstructed luma block. This is referred to as aninter-components reference. In this embodiment, it is assumed that thechroma block has a size of (nTbW×nTbH), and the luma block correspondingto the chroma block has a size of (2*nTbW×2*nTbH). That is, an inputvideo sequence may be encoded in a 4:2:0 color format.

Referring to FIG. 42 , a luma region for inter-components reference ofthe chroma block may be specified (S4200).

The luma region may include at least one of a luma block or a templateblock (hereinafter referred to as a neighboring region) adjacent to theluma block. Here, the luma block may be defined as a region includingpixels pY[x][y](x=0. . . nTbW*2−1, y=0 . . . nTbH*2−1). The pixels maymean reconstructed values before the in-loop filter is applied orreconstructed values to which the in-loop filter is applied.

The neighboring region may include at least one of a left neighboringregion, a top neighboring region, or a top-left neighboring region. Theleft neighboring region may be set as a region including pixels pY[x][y](x=−1 . . . −3, y=0 . . 2*numSampL−1). The setting may be performed onlywhen the value of numSampL is greater than 0. The top neighboring regionmay be set as a region including pixels pY[x][y] (x=0 . . .2*numSampT−1, y=−1 . . . −3). The setting may be performed only when thevalue of numSampT is greater than 0. The top-left neighboring region maybe set as a region including at least one of pixel pY[x][y](x=−1 . . .−3, y=−1, . . . −3). The setting may be performed only when the top-leftregion of the luma block is available.

The above-described variables numSampL and numSampT may be determinedbased on the intra prediction mode of the current block.

For example, when the intra prediction mode of the current block isINTRA_LT_CCLM, the variables may be derived based on Equation 9. Here,INTRA_LT_CCLM may mean a mode in which inter-component reference isperformed based on left and top neighboring regions to the currentblock.

numSampT=availT? nTbW: 0 numSampL=availL? nTbH: 0   [Equation 9]

According to Equation 9, numSampT is derived as nTbW when the topneighboring region to the current block is available. Otherwise,numSampT may be derived as 0. Similarly, numSampL is derived as nTbHwhen the left neighboring region to the current block is available.Otherwise, numSampT may be derived as 0.

To the contrary, when the intra prediction mode of the current block isnot INTRA_LT_CCLM, the variables may be derived based on Equation 10below.

numSampT=(availT && predModeIntra==INTRA_T_CCLM)? (nTbW+numTopRight):0numSampL=(availL && predModelntra==INTRA_L_CCLM)? (nTbH+numLeftBelow): 0  [Equation 10]

In Equation 10, INTRA _T_CCLM may refer to a mode in whichinter-components reference is performed based on a top neighboringregion to the current block. INTRAL_CCLM may mean a mode in whichinter-components reference is performed based on a left neighboringregion to the current block. numTopRight may mean the number of all orsome pixels belonging to a top-right neighboring region to the chromablock. Some pixels may refer to available pixels among pixels belongingto the lowest pixel row of the corresponding region. In an availabilitydetermination, whether the pixels are available may determinedsequentially in a left to a right direction. This process may beperformed until an unavailable pixel is found. numLeftBelow may mean thenumber of all or some pixels belonging to a bottom-left neighboringregion to the chroma block. Some pixels may refer to available pixelsamong pixels belonging to the rightmost pixel line (column) of thecorresponding region. Likewise, in the availability determination,whether pixels are available may be determined sequentially in a top toa bottom direction. This process may be performed until an unavailablepixel is found.

The aforementioned step of specifying the luma region may furtherinclude a process of downsampling the specified luma region.

The downsampling may include at least one of 1. Downsampling of the lumablock, 2. Downsampling of the left neighboring region to the luma block,or 3. Downsampling of the top neighboring region to the luma block. Thiswill be described in detail below.

1. Downsampling of Luma Block

Embodiment 1

The pixel pDsY[x][y] (x=0 . . . nTbW−1, y=0 . . . nTbH−1) of thedownsampled luma block may be derived based on a corresponding pixelpY[2*x][2*y] of the luma block and the neighboring pixel. Theneighboring pixel may mean at least one of a left neighboring pixel, aright neighboring pixel, a top neighboring pixel, or a bottomneighboring pixel to the corresponding pixel. For example, the pixelpDsY[x][y] may be derived based on Equation 11 below.

pDsY[x][y]=(pY[2*x][2*y−1]+pY[2*x−1][2*y]+4*pY[2*x][2*y]+pY[2*x+1][2*y]+pY[2*x][2*y+1]+4)<<3  [Equation 11]

However, there may be a case where the left/top neighboring region tothe current block are not available. When the left neighboring region tothe current block is not available, the pixel pDsY[0][y] (y=1 . . .nTbH−1) of the down-sampled luma block may be derived based on thecorresponding pixel pY[0][2*y] of the luma block and the neighboringpixel of the corresponding pixel. The neighboring pixel may mean atleast one of the top neighboring pixel or the bottom neighboring pixelto the corresponding pixel. For example, pixel pDsY[0][y] (y=1 . . .nTbH−1) may be derived based on Equation 12 below.

pDsY[0][y]=(pY[0][2*y−1]+2*pY[0][2*y]pY[0][2*y+1]+2)>>2   [Equation 12]

When the top neighboring region to the current block is not available,the pixel pDsY[x][0] (x=1 . . . nTbW−1) of the downsampled luma blockmay be derived based on the corresponding pixel pY[2*x][0] of the lumablock and the neighboring pixel of the corresponding pixel. Theneighboring pixel may mean at least one of the left neighboring pixel orthe right neighboring pixel to the corresponding pixel. For example, thepixel pDsY[x][0] (x=1 . . . nTbW−1) may be derived based on Equation 13below.

pDsY[x][0]=(pY[2*x−1][0]+2*pY[2*x][0]+pY[2*x+1][0]+2)>>2   [Equation 13]

The pixel pDsY[0][0] of the downsampled luma block may be derived basedon the corresponding pixel pY[0][0] of the luma block and/or aneighboring pixel of the corresponding pixel. The position of theneighboring pixel may vary depending on whether left/top neighboringregions to the current block are available.

For example, when the left neighboring region is available but the topneighboring region is not available, pDsY[0][0] may be derived based onEquation 14 below.

pDsY[0][0]=(pY[−1][0]+2*pY[0][0]pY[1][0]+2)>>2   [Equation 14]

To the contrary, when the left neighboring region is not available, butthe top neighboring region is available, pDsY[0][0] may be derived basedon Equation 15 below.

pDsY[0][0]=(pY[0][−1]+2*pY[0][0]+pY[0][1]+2)>>2   [Equation 15]

In another example, when both the left and top neighboring regions arenot available, pDsY[0][0] may be set as the corresponding pixel pY[0][0]of the luma block.

Embodiment 2

The pixel pDsY[x][y] (x=0 . . . nTbW−1, y=0 . . . nTbH−1) of thedownsampled luma block may be derived based on the corresponding pixelpY[2*x][2*y] of the luma block and the neighboring pixel of thecorresponding pixel. The neighboring pixel may mean at least one of abottom neighboring pixel, a left neighboring pixel, a right neighboringpixel, a bottom-left neighboring pixel, or a bottom-right neighboringpixel to the corresponding pixel. For example, the pixel pDsY[x][y] maybe derived based on Equation 16 below.

pDsY[x][y]=(pY[2*x−1][2*y]+pY[2*y1]+2*pY[2*x][2*y]2*pY[2*x][2*y+1]+pY[2*x+1][2*y]+pY[2*x+1][2*y+1]+4)>>3  [Equation 16]

However, when the left neighboring region to the current block is notavailable, the pixel pDsY[0][y] (y=0 . . . nTbH−1) of the downsampledluma block may be derived based on the corresponding pixel pY[0][2*y] ofthe luma block and a bottom neighboring pixel thereto. For example, thepixel pDsY[0][y] (y=0 . . . nTbH−1) may be derived based on Equation 17below.

pDsY[0][y]=(pY[0][2*y]+pY[0][2*y+1]+1)>>1   [Equation 17]

Downsampling of the luma block may be performed based on one ofEmbodiments 1 and 2 as described above. Here, one of Embodiments 1 and 2may be selectively performed based on a pre-determined flag. The flagmay indicate whether the downsampled luma pixel has the same position asthat of the original luma pixel. For example, when the flag is a firstvalue, the downsampled luma pixel has the same position as that of theoriginal luma pixel. To the contrary, when the flag is a second value,the downsampled luma pixel has the same position as that of the originalluma pixel in the horizontal direction, but has a position shifted byhalf pel in the vertical direction.

2. Downsampling of Left Neighboring Region to Luma Block

Embodiment 1

The pixel pLeftDsY[y] (y=0 . . . numSampL−1) of the downsampled leftneighboring region may be derived based on the corresponding pixelpY[−2][2*y] of the left neighboring region and a neighboring pixel ofthe corresponding pixel. The neighboring pixel may mean at least one ofa left neighboring pixel, a right neighboring pixel, a top neighboringpixel, or a bottom neighboring pixel to the corresponding pixel. Forexample, the pixel pLeftDsY[y] may be derived based on Equation 18below.

pLeftDsY[y]=(pY[−2][2*y−1]+pY[−3][2*y]+4*pY[−2][2*y]+pY[−1][2y]+pY[−2][2*y+1]+4)>>3  [Equation 18]

However, when the top-left neighboring region to the current block isnot available, the pixel pLeftDsY[0] of the downsampled left neighboringregion may be derived based on the corresponding pixel pY[−2][0] of theleft neighboring region and a neighboring pixel of the correspondingpixel. The neighboring pixel may mean at least one of the leftneighboring pixel or a right neighboring pixel to the correspondingpixel. For example, the pixel pLeftDsY[ 0] may be derived based onEquation 19 below.

pLeftDsY[0]=(pY[−3][0]+2*pY[−2][0]+pY[−1][0]+2)>>2   [Equation 19]

Embodiment 2

The pixel pLeftDsY[y] (y=0 . . . numSampL−1) of the downsampled leftneighboring region may be derived based on the corresponding pixelpY[−2][2*y] of the left neighboring region and a neighboring pixelaround the corresponding pixel. The neighboring pixel may mean at leastone of a bottom neighboring pixel, a left neighboring pixel, a rightneighboring pixel, a bottom-left neighboring pixel, or a bottom-rightneighboring pixel to the corresponding pixel. For example, the pixelpLeftDsY[y] may be derived based on following Equation 20.

pLeftDsY[y]=(pY[−1][2*y]+pY[−1][2*y+1]+2*pY[−2][2*y]2*pY[−2][2*y+1]+pY[−3][2*y]+pY[−3][2*y+1]+4)>>3  [Equation 20]

Similarly, downsampling of the left neighboring region may be performedbased on one of Embodiments 1 and 2 as described above. Here, one ofEmbodiments 1 and 2 may be selected based on a pre-determined flag. Theflag indicates whether the downsampled luma pixel has the same positionas that of the original luma pixel. This is the same as described above.

Downsampling of the left neighboring region may be performed only whenthe numSampL value is greater than 0. When the numSampL value is greaterthan 0, it may mean that the left neighboring region to the currentblock is available, and the intra prediction mode of the current blockis INTRA_LT_CCLM or INTRA_L_CCLM.

3. Downsampling of Top Neighboring Region to Luma Block

Embodiment 1

The pixel pTopDsY[x] (x=0 . . . numSampT−1) of the downsampled topneighboring region may be derived in consideration of whether the topneighboring region belongs to a CTU different from a CTU to which theluma block belongs.

When the top neighboring region belongs to the same CTU as the lumablock, the pixel pTopDsY[x] of the downsampled top neighboring regionmay be derived based on the corresponding pixel pY[2*x][−2] of the topneighboring region and a neighboring pixel of the corresponding pixel.The neighboring pixel may mean at least one of a left neighboring pixel,a right neighboring pixel, a top neighboring pixel, or a bottomneighboring pixel to the corresponding pixel. For example, the pixelpTopDsY[x] may be derived based on Equation 21 below.

pTopDsY[x]=(pY[2*x][−3]+pY[2*x−1][−2]+4*pY[2*x][−2]pY[2*x+1][−2]+pY[2*x][−1]+4)>>3   [Equation 21]

To the contrary, when the top neighboring region belongs to a CTUdifferent from the luma block, the pixel pTopDsY[x] of the downsampledtop neighboring region may be derived based on the corresponding pixelpY[2*x][−1] of the top neighboring region and a neighboring pixel of thecorresponding pixel. The neighboring pixel may mean at least one of theleft neighboring pixel or the right neighboring pixel to thecorresponding pixel. For example, the pixel pTopDsY[ x] may be derivedbased on Equation 22 below.

pTopDsY[x]=(pY[2*x−1][−1]+2*pY[2*x][−1]+pY[2*x+1][−1]+2) >>2   [Equation22]

Alternatively, when the top-left neighboring region to the current blockis not available, the neighboring pixel may mean at least one of the topneighboring pixel or the bottom neighboring pixel to the correspondingpixel. For example, the pixel pTopDsY[ 0] may be derived based onEquation 23 below.

pTopDsY[0]=(pY[0][−3]+2*pY[0][−2]+pY[0][−]+2)>>2   [Equation 23]

Alternatively, when the top-left neighboring region to the current blockis unavailable and the top neighboring region belongs to a CTU differentfrom the luma block, the pixel pTopDsY[0] may be set as the pixelpY[0][−1] of the top neighboring region.

Embodiment 2

The pixel pTopDsY[x] (x=0 . . . numSampT−1) of the downsampled topneighboring region may be derived in consideration of whether the topneighboring region belongs to a CTU different from the luma block.

When the top neighboring region belongs to the same CTU as the lumablock, the pixel pTopDsY[x] of the downsampled top neighboring regionmay be derived based on the corresponding pixel pY[2*x][−2] of the topneighboring region and a neighboring pixel of the corresponding pixel.The neighboring pixel may mean at least one of a bottom neighboringpixel, a left neighboring pixel, a right neighboring pixel, abottom-left neighboring pixel, or a bottom-right neighboring pixel tothe corresponding pixel. For example, the pixel pTopDsY[x] may bederived based on Equation 24 below.

pTopDsY[x]=(pY[2*x−1][−2]+pY[2*x−1][−1]+2*pY[2*x][−1]+pY[2*x+1][−2]+pY[2*x+1][−1]+4)>>3   [Equation 24]

To the contrary, when the top neighboring region belongs to a CTUdifferent from the luma block, the pixel pTopDsY[x] of the downsampledtop neighboring region may be derived based on the corresponding pixelpY[2*x][−1] of the top neighboring region and a neighboring pixel of thecorresponding pixel. The neighboring pixel may mean at least one of theleft neighboring pixel or the right neighboring pixel to thecorresponding pixel. For example, the pixel pTopDsY[ x] may be derivedbased on Equation 25 below.

pTopDsY[x 1802 ]=(pY[2*x−1][−1]+2*pY[2*x][−1]+pY[2*x+1][−1]+2) >>2  [Equation 25]

Alternatively, when the top-left neighboring region to the current blockis not available, the neighboring pixel may mean at least one of the topneighboring pixel or the bottom neighboring pixel to the correspondingpixel. For example, the pixel pTopDsY[0] may be derived based onEquation 26 below.

pTopDsY[0]=(pY[0][−2]+pY[0][−1]+1)>>1   [Equation 26]

Alternatively, when the top-left neighboring region to the current blockis unavailable and the top neighboring region belongs to a CTU differentfrom the luma block, the pixel pTopDsY[0] may be set as pixel pY[0][−1]of the top neighboring region.

In the similar manner, downsampling of the top neighboring region may beperformed based on one of Embodiments 1 and 2 as described above. Here,one of Embodiments 1 and 2 may be selected based on a pre-determinedflag. The flag indicates whether the downsampled luma pixel has the sameposition as that of the original luma pixel. This is the same asdescribed above.

In one example, downsampling of the top neighboring region may beperformed only when the numSampT value is greater than 0. When thenumSampT value is greater than 0, it may mean that the top neighboringregion to the current block is available, and the intra prediction modeof the current block is INTRA_LT_CCLM or INTRA_T_CCLM.

A parameter for inter-components reference of the chroma block may bederived (S4210).

The parameter may be determined in consideration of the intra predictionmode of the current block. That is, the related parameters may bederived in a variable manner depending on whether the intra predictionmode of the current block is INTRA_LT_CCLM, INTRA_L_CCLM, orINTRA_T_CCLM.

The parameter may be calculated based on a linear change between aneighboring region to a luma block and a neighboring region to a chromablock. The parameter may be derived using at least one of a pixel of aluma region or a pixel of top/left neighboring regions to a chromablock. Here, the luma region may include at least one of the luma blockor the top/left neighboring regions to the luma block. The luma regionmay mean a region to which the aforementioned downsampling is applied.

The chroma block may be predicted based on the luma region and theparameter (S4220).

For example, the parameter may be applied to the pixel of thedownsampled luma block to generate a prediction block of the chromablock.

FIG. 43 shows a method of determining delta motion information in anembodiment to which the present disclosure is applied.

For convenience of description, it is assumed that an initial value ofdelta motion information is set to 0 in the decoder.

The decoder may determine an optimal reference block by performing asearching operation based on the position of the reference block for thecurrent block. Here, the reference block for the current block may meana region indicated by the initial motion information. The decoder mayupdate delta motion information based on the position of the optimalreference block. Here, the reference block may include L0/L1 referenceblocks. The L0 reference block may refer to a reference block belongingto a reference picture of the reference picture list 0 (L0), while theL1 reference block may mean a reference block belonging to the referencepicture of the reference picture list 1 (L1).

Referring to FIG. 43 , the decoder may determine a search region forimproving motion information (S4300).

The search region may be determined as a region including at least oneof an reference block or an adjacent region to the reference block. Inthis case, a position of a top-left sample of the reference block mayact as a reference position for the search. The search region may bedetermined in each of the L0 direction and the L1 direction. Theadjacent region may mean a region extended by N number of sample linesfrom a boundary of the reference block. Here, N may be an integer of 1,2, 3 or greater.

The adjacent region may be located in at least one direction of left,top, right, bottom, top-left, bottom-left, top-right, or bottom-right ofthe reference block. Here, when the current block is WxH, the searchregion may be expressed as (W+2N)×(H+2N). However, in order to reducethe complexity of the improvement process, the adjacent region may belimited to only some of above-described directions. For example, theadjacent region may be limited to the left and top adjacent regions tothe reference block or may be limited to the right and bottom adjacentregions to the reference block.

The number of sample lines (N) may be a fixed value pre-defined in thedecoder or may be determined variably in consideration of the attributeof the block. Here, the attribute of the block may mean a blocksize/shape, block position, inter prediction mode, component type, andthe like. The block position may mean whether the reference block islocated on the boundary of a picture or a pre-determined fragmentregion. The fragment region may mean a tile, a coding tree blockcolumn/row, or a coding tree block. For example, based on the attributeof the block , the number of sample lines may be selectively determinedas 0, 1 or 2.

Referring to FIG. 43 , the decoder may determine a sum of absolutedifference (SAD) at each search position within the search region(S4310).

Specifically, a list (hereinafter referred to as an SAD list) specifyingthe SAD at each search position within the search region may bedetermined. Here, the SAD list may include a plurality of SADcandidates. The number of SAD candidates constituting the SAD list maybe M. M may be an integer greater than or equal to 2. A maximum value ofM may be limited to 9.

The SAD candidate may be determined based on the SAD value between theL0 block and the L1 block. Here, the SAD value may be calculated basedon all samples belonging to the L0 and L1 blocks, or may be calculatedbased on some of samples belonging to the L0 and L1 blocks. Here, somesamples refer to a sub-block of each of the L0 and L1 blocks. In thiscase, at least one of a width or a height of the sub-block may be 1/2 ofa width or a height of the L0/L1 blocks. That is, each of the L0 and L1blocks may have the size of W×H, and said some samples may refer to asub-block having a size of W×H/2, W/2×H or W/2×H/2. Here, when a size ofsaid some samples has W×H/2, said some samples may refer to an topsub-block (or a bottom sub-block) in each of the L0 and L1 blocks. Whena size of said some samples is W/2×H, said some samples may refer to aleft sub-block (or a right sub-block) in each of the L0 and L1 blocks.When a size of said some samples is W/2×H/2, said some samples may referto a top-left sub-block in each of the L0 and L1 blocks. However, thepresent disclosure are not limited thereto. Alternatively, said somesamples may be defined as a group of even-numbered or odd-numberedsample lines (vertical or horizontal direction) of each of the L0 and L1blocks.

The position of the L0 block may be determined based on the position ofthe L0 reference block for the current block and a pre-determinedoffset. The offset may mean a disparity vector between the position ofthe L0 reference block and the search position. That is, the searchposition may be a position shifted by p in the x-axis direction and q inthe y-axis direction from the position (x0,y0) of the L0 referenceblock. Here, each of p and q may be at least one of −1, 0, or 1. Here,the disparity vector created based on a combination of p and q may meanthe offset. The position of the L0 block may be determined as a positionshifted by (p,q) from the position (x0,y0) of the L0 reference block. Asize (or absolute value) of each of p and q is 0 or 1. However, thepresent disclosure is not limited thereto. For example, a size of eachof p and q may be 2, 3, or greater.

The offset may include at least one of a non-directional offset (0, 0)or a directional offset. The directional offset may include an offset inat least one of left, right, top, bottom, top-left, top-right,bottom-left, or bottom-right direction. For example, the directionaloffset may include at least one of (−1,0), (0,1), (0,−1), (0,1),(−1,−1), (−1,1), (1,−1) or (1,1).

In the same manner, the position of the L1 block may be determined basedon the position of the L1 reference block for the current block and apre-determined offset. Here, the offset of the L1 block may bedetermined based on the offset of the L0 block. For example, when theoffset of the L0 block is (p,q), the offset of the L1 block may bedetermined as (−p, −q).

Information on the size and/or direction of the offset as aforementionedmay be pre-defined in the decoder, or may be encoded by the encoder andmay be signaled to the decoder. The information may be variablydetermined in consideration of the attribute of the block asaforementioned.

In an example, the offset may be defined as shown in Table 3 below.

TABLE 3 I 0 1 2 3 4 5 6 7 8 dX[i] −1 0 1 −1 0 1 −1 0 1 dY[i] −1 −1 −1 00 0 1 1 1

Table 3 defines an offset for determining the search position at eachindex i. However, Table 3 does not limit the position of the offsetcorresponding to the index i. Rather, the position of the offset at eachindex may be different from those in Table 3. The offset according toTable 3 may include the non-directional offset (0, 0) and the eightdirectional offsets as aforementioned.

In this case, a 0-th SAD candidate may be determined based on theposition (x,y) of the reference block and the offset (−1,−1).Specifically, a position shifted by the offset (−1,−1) from the position(x0,y0) of the L0 reference block may be set as the search position. AW×H block including the search position as an top-left sample may bedetermined as the L0 block.

Similarly, the position shifted by the offset (1,1) from the position(x1,y1) of the L1 reference block may be set as the search position. AWxH block including the search position as an top-left sample may bedetermined as the L1 block. The 0-th SAD candidate may be determined bycalculating the SAD between the L0 block and the L1 block.

Via the above-described process, first to eighth SAD candidates may bedetermined, and the SAD list including the 9 SAD candidates may bedetermined.

Table 3 does not limit the number of offsets used for improving motioninformation. Only k offsets of the 9 offsets may be used. Here, k may beany value from 2 to 8. For example, in Table 3, three offsets such as[0,4,8], [1,4,7], [2,4,6], [3,4,5], etc. may be used, or four offsetssuch as [0,1,3,4], [4,5,7,8], etc. may be used, or six offsets such as[0,1,3,4,6,7,], [0,1,2,3,4,5,], etc. may be used.

In an example, the offset may be defined in a manner as shown in Table 4below. That is, the offset may be composed only of the non-directionaloffset (0,0), offsets (−1,0) and (1,0) in the horizontal direction andoffsets (0,−1) and (0,1) in the vertical direction.

TABLE 4 i 0 1 2 3 4 dX[i] 0 −1 0 1 0 dY[i] −1 0 0 0 1

In this way, the size and/or shape of the search region asabove-described may be variably determined based on the size and/ornumber of offsets. The search region may have a geometric shape otherthan a square shape.

Referring to FIG. 43 , the delta motion information may be updated basedon the SAD at each search position (S4320).

Specifically, one of a first method and a second method for improvingthe motion information about the current block may be selected based onthe SAD candidate of the SAD list as above-described.

The selection may be performed based on a comparison result between areference SAD candidate and a pre-determined threshold (Embodiment 1).

The reference SAD candidate may mean a SAD candidate corresponding tothe offset (0,0). Alternatively, the reference SAD candidate may mean aSAD candidate corresponding to a position of a reference block or areference position changed using the first method to be described later.

The threshold may be determined based on at least one of the width (W)or height (H) of the current block or the reference block. For example,the threshold may be determined as W*H, W*(H/2), (W/2)*H, 2*W*H, 4*W*H,and the like.

When the reference SAD candidate is greater than or equal to thethreshold, the delta motion information may be updated based on thefirst method. Otherwise, the delta motion information may be updatedbased on the second method.

(1) Regarding First Method

A SAD candidate with the minimum value among the SAD candidates in theSAD list may be identified. The identification method will be describedlater. The delta motion information may be updated based on the offsetcorresponding to the identified SAD candidate.

Then, the search position corresponding to the identified SAD candidatemay be changed to a reference position for search. The SAD listdetermination and/or the identification of the SAD candidate having theminimum value as described above may be re-performed based on thechanged reference position. Duplicate descriptions thereof will beomitted. Then, the delta motion information may be updated based on there-performing result.

(2) Regarding Second Method

Delta motion information may be updated based on a parameter calculatedusing all or some of the SAD candidates in the SAD list.

The parameter may include at least one of a parameter dX related to anx-axis direction or a parameter dY related to a y-axis direction. Forexample, the parameter may be calculated as follows. For convenience ofdescription, the calculation will be described based on Table 3 asdescribed above.

When the sum of sadList[3] and sadList[5] is equal to (sadList[4]<<1),dX may be set to 0. Here, sadList[3] may mean a SAD candidatecorresponding to the offset (−1,0), sadList[5] may mean a SAD candidatecorresponding to the offset (1,0), and sadList[4] may mean a SADcandidate corresponding to the offset (0,0).

When the sum of sadList[3] and sadList[5] is greater than(sadList[4]<<1), dX may be determined based on Equation 27 below.

dX=((sadList[3]sadList[5])<<3)/(sadList[3]+sadList[5](sadList[4]<<1))  [Equation 27]

When the sum of sadList[1] and sadList[7] is equal to (sadList[4]<<1),dY may be set to 0. Here, sadList[1] may refer to a SAD candidatecorresponding to the offset (0,−1), sadList[7] may refer to a SADcandidate corresponding to the offset (0,1), and sadList[4] may mean aSAD candidate corresponding to the offset (0,0).

When the sum of sadList[1] and sadList[7] is greater than(sadList[4]<<1), dY may be determined based on Equation 28 below.

dY=((sadList[1]sadList[7])<<3)/(sadList[1]+sadList[7](sadList[4]<<1))  [Equation 28]

Based on the calculated parameter, the delta motion information may beupdated.

Alternatively, the above-described selection may be performed based onwhether the reference SAD candidate is a SAD candidate having a minimumvalue among the SAD candidates included in the SAD list (Embodiment 2).

For example, when the SAD candidate having the minimum value among theSAD candidates in the SAD list is not the reference SAD candidate, thedelta motion information may be updated based on the first method.Otherwise, the delta motion information may be updated based on thesecond method. The first method and the second method are the same asdescribed above, and thus detailed descriptions thereof will be omitted.

Alternatively, the above-described selection may be performed based on acombination of the first and second methods (Embodiment 3).

For example, when the reference SAD candidate is smaller than thethreshold, the delta motion information may be updated using theparameter according to the second method as described above.

However, when the reference SAD candidate is greater than or equal tothe threshold, the SAD candidate with the minimum value among the SADcandidates in the SAD list may be identified. When the identified SADcandidate is the reference SAD candidate, the delta motion informationmay be updated using the parameter according to the second method asdescribed above. To the contrary, when the identified SAD candidate isnot the reference SAD candidate, the delta motion information may beupdated according to the first method as described above.

A method for identifying the SAD candidate with the minimum value willbe described. The SAD candidates in the SAD list may be grouped into 2,3, or more groups. Hereinafter, for convenience of description, anexample where the SAD candidates are grouped into two groups will bedescribed.

A plurality of SAD candidates may be grouped into a first group and asecond group. Each group may include at least two SAD candidates amongSAD candidates in the SAD list. However, the group may be configuredonly so that the reference SAD candidate is not included therein. Theminimum operation may be apply to each group to extract the SADcandidate with the minimum value in each group.

A SAD candidate (hereinafter referred to as a temporary SAD candidate)having a minimum value among the SAD candidate extracted from the firstgroup and the SAD candidate extracted from the second group may beextracted again.

Then, the SAD candidate having the minimum value may be identified basedon the comparison result between the temporary SAD candidate and thereference SAD candidate. For example, when the temporary SAD candidateis smaller than the reference SAD candidate, the temporary SAD candidatemay be identified as the smallest SAD candidate in the SAD list. To thecontrary, when the temporary SAD candidate is greater than or equal tothe reference SAD candidate, the reference SAD candidate may beidentified as the smallest SAD candidate in the SAD list.

The motion derivation method (especially, the motion informationimprovement method) in the decoder as described above may be performedon a sub-block basis and based on the block size.

For example, when the size of the current block is greater than or equalto the pre-determined threshold size, the motion information improvementmethod may be performed on the threshold size basis. To the contrary,when the size of the current block is smaller than the pre-determinedthreshold size, the motion information improvement method may beperformed on the current block basis. Here, the threshold size may bedetermined such that at least one of the width or the height of theblock is a T value. The T value may be an integer of 16, 32, 64 orgreater.

The motion derivation method (especially, the motion informationimprovement method) in the decoder as described above may be performedin a limited manner based on the block size, distance the between thecurrent picture and the reference picture, the inter prediction mode,the prediction direction, or the unit or resolution of the motioninformation.

For example, the motion information improvement method may be performedonly when any one of the width or height of the current block is greaterthan or equal to 8, 16, or 32. Alternatively, the motion informationimprovement method may be applied only when the width and height of thecurrent block are greater than or equal to 8, 16, or 32. Alternatively,the motion information improvement method may be applied only when thearea of the current block or the number of samples thereof is greaterthan or equal to 64, 128, or 256.

The motion information improvement method may be applied only when adifference between POCs of the current picture and L0 the referencepicture and a difference between POCs of the current picture and L1 thereference picture are equal to each other.

The improvement method may be applied only when the inter predictionmode of the current block is the merge mode. When the inter predictionmode of the current block is other modes (e.g., affine mode, AMVP mode,etc.), the motion information improvement method may not be applied.

The motion information improvement method may be applied only when thecurrent block is subjected to the bi-directional prediction. Further,the motion information improvement method may be applied only when theunit of the motion information is an integer pel. The motion informationimprovement method may be applied only when the unit of the motioninformation is equal to or smaller than a quarter pel or a half-pel.

Various embodiments of the present disclosure are intended not to listall possible combinations of embodiments but to illustraterepresentative aspects of the present disclosure. The features invarious embodiments may be applied independently or may be applied incombination of two or more thereof.

Further, various embodiments of the present disclosure may beimplemented in hardware, firmware, software, or a combination thereof.In the hardware based implementation, at least one of ASICs (ApplicationSpecific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs(Digital Signal Processing Apparatuss), PLDs (Programmable LogicApparatuss), FPGAs (Field Programmable Gate Arrays), general processors,controllers, a microcontroller, a microprocessor, or the like may beused.

The scope of the present disclosure may include software ormachine-executable instructions (e.g., operating system, application,firmware, programs, etc.) to cause the operations of the methodaccording to the present disclosure to be executed on a apparatus or acomputer, and a non-transitory computer-readable medium storing thereinsuch software or instructions as executable on the apparatus or thecomputer.

INDUSTRIAL AVAILABILITY

The present disclosure may be used to encode/decoding the video data.

What is claimed is:
 1. A method of decoding video data, the methodcomprising: determining initial motion information of a current block;determining delta motion information of the current block; modifying theinitial motion information of the current block by using the deltamotion information; and performing motion compensation of the currentblock by using the modified motion information.
 2. The method of claim1, wherein determining the delta motion information comprises:determining a search region for modifying the initial motioninformation; generating a sum of absolute difference (SAD) list from thesearch region, the SAD list including SAD candidates, each of the SADcandidates corresponding to each of search positions in the searchregion; and updating the delta motion information based on the SADcandidates in the SAD list.
 3. The method of claim 2, wherein an SADcandidate is determined as an SAD value between an L0 block and an L1block, and wherein the SAD value is calculated based on only samplesbelonging to one of odd sample lines or even sample lines in the L0 andL1 blocks.
 4. The method of claim 3, wherein a position of the L0 blockis determined based on a position of a L0 reference block of the currentblock and a predetermined offset, and wherein the offset includes atleast one of a non-directional offset or a directional offset.
 5. Themethod of claim 4, wherein updating the delta motion information isperformed based on a comparison result between a reference SAD candidateand a predetermined threshold value, and wherein the reference SADcandidate means an SAD candidate corresponding to the non-directionaloffset.
 6. The method of claim 5, wherein the threshold value isdetermined as a product of width and height of the current block.
 7. Themethod of claim 6, wherein, in response to the reference SAD candidatebeing greater than or equal to the threshold value, an SAD candidatehaving a minimum value among the SAD candidates in the SAD list isidentified, and wherein the delta motion information is updated based onan offset corresponding to the identified SAD candidate.
 8. The methodof claim 7, wherein the modification of the initial motion informationis selectively performed in consideration of at least one of a size ofthe current block, a distance between a current picture and a referencepicture of the current block, an inter prediction mode, or a predictiondirection.
 9. A method of encoding video data, the method comprising:determining initial motion information of a current block; determiningdelta motion information of the current block; modifying the initialmotion information of the current block by using the delta motioninformation; and performing motion compensation of the current block byusing the modified motion information.
 10. A non-transitory computerreadable medium for storing a bitstream generated by a video encodingmethod, the video encoding method comprising: determining initial motioninformation of a current block; determining delta motion information ofthe current block; modifying the initial motion information of thecurrent block by using the delta motion information; and performingmotion compensation of the current block by using the modified motioninformation.